In an exclusive nine-part dialogue with an imaginary eBay Architect, we present an accessible discussion of the REST vs. SOA issue.
Although eBay have what they call a 'REST' interface, it is, in fact, a STREST interface, and only works for a few of the many function calls that they make available via SOAP.
In this dialogue series, I argue the case for eBay to adopt a truly REST approach to their integration API.
Part 9: Web Objects Ask, They Never Tell
eBay Architect: You've pushed a lot of responsibility up to the business logic and away from the distribution technologies - away from the HTTP level.
As I understand it, you only want to use HTTP to implement a distributed Observer Pattern, where clients can become servers?
Duncan Cragg: Indeed. REST can be symmetric when used in integration outside of the Web. Further, these server-clients can have resources that do their own GET-ing, with POST callbacks, in order to Observe other resources. I've given many examples of this style.
eA: So how do you program such a distributed system at the business logic level?
DC: Well you need to be able to easily express that the state of a resource depends on the latest state - the intentions and declarations - of other resources it is Observing.
My vision is that you would express this business logic in a simple, powerful and expressive declarative language.
eA: Hmmm.. Just try getting all declarative on someone who just wants his share prices on time! No-one in business understands such abstract concepts.
DC: Oh no? The favourite programming tool of business is the spreadsheet!
eA: Well I've seen some pretty ghastly examples of those two crafts!
And many failed attempts to allow non-programmers or business analysts to program directly.
DC: True, but that doesn't change the fact that most non-programmers can only think declaratively - they know What they want, but they haven't got much of a clue How to get it.
Business people talk business rules and business data.
eA: We just can't program with them!
DC: Well, perhaps we just haven't found the best discipline of pragmatic formalisms and methodology that will allow those rules to easily become programs.
It's a well-supported claim, though, that declarative programming is simpler, clearer and more productive than imperative.
eA: OK - so how would you actually define these business rules?
DC: You could use a rules engine, or a DSL engine, or even XSLT - if that works for you. There are many ways to transform XML.
eA: How would you do it?
DC: I would like to have an XML rewriting and templating system: "if this XPath or XML template matches Observed XML resource A, this one matches resource B and this one matches myself, rewrite this and that bit of myself with these bits from A and B".
Web Objects Ask, They Never Tell
eA: So you want to go around re-creating huge XML documents all the time?
DC: Who said anything about huge? This is another Web thing that doesn't necessarily apply to us in REST integration: we don't need the equivalent of the giant, monolithic HTML page. We can work in much smaller chunks.
I'd even just call them 'objects' rather than resources. Or 'Web objects', if you like.
They could be little XHTML carriers of Microformats.
eA: I thought we'd given up on distributing fine-grained objects back in the CORBA days?
DC: Ah, but this isn't trying to transparently distribute zillions of method calls in an RPC model.
This is about optimising state transfer. Only send what you need, cache where you can, push when something changes. Separate your data by rate of change, timeliness, cacheability.
eA: So these Web objects don't have any methods? Because that would be RPC when distributed?
DC: Exactly. In REST integration, the Web objects Ask, they never Tell! These Web objects are reactive: Asking for public state, not Telling each other what to do.
eA: You mean the opposite of Tell Don't Ask?
DC: Yes. As an object, you don't Tell another object How to do something, you Ask for What you want by either simply Observing its public state or by it Observing yours, then letting it decide How to evolve by itself. You then watch it and react or interact.
It's the "imperative to declarative inversion": everything is turned upside-down or inside-out when you distribute things this way!
eA: Oh yes, your "inevitable inversion" thing.
DC: Another indication of this inversion from the imperative object-oriented world to the declarative ROA world is how the derided 'train wrecks' of object-orientation now become the essential XPaths of the object Web.
You could say we have no methods, only 'getters', and XPath 'train wrecks' are encouraged!
eA: Doesn't sound too safe to me - it breaks encapsulation, doesn't it?
DC: Well, it's actually safe to dig around, since the data is held in shape by a stable, open, public schema. You're expected to go traversing the tree.
And you get excellent encapsulation since Web objects are total masters of their own destiny: they privately control the evolution of that public state.
You very much retain the value of 'What not How'; in fact, in a much better-defined way since it's fundamentally baked in to the programming model.
Hyperdata as the Engine of Object State
eA: But when you write 'train wrecks' you often end up jumping from object to object. How do your object Web XPaths do that, assuming they want to?
DC: Hyperdata of course! Links to links around the Web. Objects can have their opaque UUIDs or GUID object handles encoded into their URIs. Then objects can be wired up with XHTML links.
eA: So now your XPath transparently jumps these links?!
DC: Actually, yes! That would then allow us to dynamically break up data into more manageable chunks without breaking the XPaths that traverse it.
Also, with this approach, you still get to drill down to data like in transparent URI paths, but you now use XPaths that are properly a part of the content layer, jumping transparently over those opaque inter-object URIs.
eA: Would you use these 'jumping XPaths' in the rewrite rules you said you wanted?
DC: Of course. Either linear XPaths, or XPath-like XML tree templates on the left-hand side of a rewrite rule.
You'd start a template match on yourself, then continue on to match other objects by jumping over links, then on and on from object to object.
eA: So I suppose any such jump to another object means you then need to start Observing it, right?
DC: Exactly. And if hyperlinks are the only way you can find other objects to Observe, it brings us to the following:
Your object's next public state depends only on its current public state and the states of those objects that are visible to it through hyperlinks.
eA: Sounds a bit like the "Hypertext As The Engine Of Application State" constraint of REST.
DC: Exactly! Except now that we're doing REST symmetrically - now that clients can be servers, too - client Application State can have its own URIs!
eA: So you could re-phrase this as the even more intimidating: "Hyperdata As The Engine Of Application Resource State"!
DC: Well - how about just "Hyperdata As The Engine Of Object State"?!
eA: So do I have to wait for this link-jumping XML rewrite engine of yours to express my business logic, in order to get "Hyperdata As The Engine Of Object State"?
DC: No, of course not! Use a nice, dynamic, XML-talking language, like Scala, and follow the same principle, perhaps using a DSL.
eA: Can't you have objects that aren't entirely dependent on others? Like those that represent external state?
DC: Of course. That's normal Web stuff. It's probably best to keep these 'pure': to have either fully-interdependent objects driven by Hyperdata, or fully externally-driven ones.
Class, Extension, Instance and Behaviour
eA: Right, so we've got these little Web-mapped XHTML Microformat objects all linked up and watching each other in an Observer Pattern. This object's state depends on that linked object's state according to rewrite rules or a DSL.
So taking this mapping to objects one last step, I presume object 'class' maps onto an XML schema, XHTML Microformat specification or other content type?
DC: Yes. A public grammar in some form. Domain or business classes only, of course, not low-level classes.
eA: So, does each standard Web object class or type have a standard set of rules guiding its evolution or behaviour?
DC: Yes. If you see something and recognise its type, you can attempt to interact with it according to its public specification, and it should, but needn't, react.
The public specification can define the expected behaviour in the RFC language of MUST and SHOULD, like AtomPub. Or it can define it in rewrite rules!
eA: So then, how do I add my own business rules to a content type or 'class', if its behaviour is standardised and meant to be stable and predictable?
DC: There are two ways you can make use of a generic standard, and both are forms of layering or abstraction:
You can 'mash up' standard component instances with declarative configuration - just using them as they are for your domain.
Alternatively, you can extend or subclass their standard structures and behaviour to become themselves more domain-specific. For example, rules can be overridden or their right-hand sides merged.
Again, think of how AtomPub can be used in any domain that looks like time-ordered lists of content; and how you can extend AtomPub without breaking unextended clients or servers.
The REST Observer Pattern
eA: I think you need a memorable name for your symmetric-REST, "Hyperdata As The Engine Of Object State" architectural style!
DC: How about the "REST Observer Pattern"?
eA: But a Pattern has to be a retro-fit to existing behaviour, and "REST Observer" is quite new.
DC: I mean REST .. [DC does double bunnie ears] "Observer Pattern"!
eA: Have you got any worked examples of the REST Observer Pattern?
DC: Will have soon...
(c) 2006-2009 Duncan Cragg
Coming soon: Worked examples of the REST Observer Pattern.
Note that the opinions of our imaginary eBay Architect don't necessarily represent or reflect in any way the official opinions of eBay or the opinions of anyone at eBay.
Indeed, I can't guarantee that the opinions of our real blogger necessarily represent or reflect in any way the official opinions of Roy Fielding...