In an exclusive nine-part dialogue with an imaginary eBay Architect, we present an accessible discussion of the REST vs. SOA issue.
Although eBay have what they call a 'REST' interface, it is, in fact, a STREST interface, and only works for a few of the many function calls that they make available via SOAP (GetSearchResults, GetItem, GetCategoryListings, etc).
In this dialogue series, I argue the case for eBay to adopt a truly REST approach to their integration API.
Part 5: The Distributed Observer Pattern
eBay Architect: So, can you summarise your argument that 'REST isn't just about reading and writing data', and explain your view on RESTful business logic?
Duncan Cragg: OK. The whole collection of related resources determines where things stand at any given time.
Resources are masters of their own destiny - guided by rules declared in the standard to which their content type conforms.
These rules, or business logic, run on notification of any declarations of the state of peer resources, or on arrival of any state via POST. Such peer states and POSTs are not commands, although it is possible to go ahead and define a special command or edit command content type.
The rules aim to satisfy the business or domain constraints on the mutual states of these resources - updating and creating resources accordingly and causing appropriate side-effects outside the server, such as financial transactions and emails.
These transformations are state driven. Even though the 'tension' in unresolved rules may be detected by events, that tension exists, not in those events as such, but in resource state.
eA: That sounds like a core difference to SOA.
DC: Indeed. It's a Resource-Oriented Architecture. And ROAs are declarative, not imperative like SOAs.
We have a world of resources declaring their current state, and resources settling into new states depending on the current state of related resources. These state changes can be driven by hard-coded resource animation logic, or by simpler, clearer, more scalable, declarative state transformation rules.
eA: Remind me of those patterns for notifying state change.
DC: Resource states are either polled via GET or actively notified via POST. Such actively POSTed state could be from a resource that also happens to be GETable, could be simply a link to such a resource, or could cause such a GETable resource to be created on the target server. Alternatively, the POSTed state could be considered too transient to record in a GETable resource, but can still trigger transformation in its target resource.
The above eBay examples used the pattern of 'server creates GETable copy of POSTed resource', and also 'second server hosts GETable copy of POST-notified resource'.
What I have described is a general programming model because, in general, such simple, declarative, transformational mechanisms are Turing Complete.
eA: I'm sure it's a novel perspective - even to RESTians! Again, do you have any high-level RESTian support for this?
DC: Any web resource that is a derivative of, or is dependent on, one or more other resources is using this approach.
Like I said before, there is an example of a similar approach by Joe Gregorio on his 'Well-Formed Web' site for alerting resources to peer resources of mutual interest.
Every time you would POST some data, consider making that data GETable and POST its URI instead, as a notification of the data existing.
eA: GETable POST data? You sure that's REST-compliant?
DC: In REST integration, things become more symmetric than in the client-server Web, or rather, the 'client-resource' Web. We can start to talk about the 'resource-resource' Web!
But anyway, we're already halfway to the symmetric resource-resource Web when we POST - not to a service, but to a URI. Resources can already both issue and receive state, which is a pretty symmetric state of affairs.
eA: I never thought of it that way - I keep forgetting that you can POST right back to a resource you just fetched.
DC: But think one step on: the POSTed data has a Content-Type but no URI!
Why not close the loop and have this POSTed data be a first-class resource (with a URI) that POSTs itself to the target. And it can itself GET that target or be POSTed to by that target in return.
That really is a Resource-Oriented Architecture. Once resources are seen as equal and active participants in RESTful integration, it becomes irrelevant whether their state is transferred by GET or by POST.
eA: I'm still having trouble with this pattern of POST just being a pro-active GET.
DC: Making POSTed data GETable more correctly moves the responsibility to the target resource to fetch the incoming resource state when its ready (rather than being bombarded by state it hasn't asked for).
Once the target is interested, updates can be POSTed directly as they happen, to prevent the target polling, or notification of an updated URI POSTed to trigger the target to re-GET the changed resource when it wants (thereby updating the caches).
eA: Hmm - makes clients look like servers..
DC: Since our 'clients' in REST integration are also 'servers' in other contexts, it is easier to set up client-side resources than on the browser-based Web. One objection to cookies on the Web is that they are state or resource that has no URI. So give your 'client' state a URI! And put any client-specific server resources on your own 'client' host.
eA: Is anyone doing this sort of thing?
DC: Well, in fact there are many examples of this POST-notification of a GETable resource already happening between web sites. Like submitting a link to your site to an indexing engine and letting it crawl (or poll) it.
Trackback pings are another example: POST a URI along with a sample of your page. And the Microformat rel-tag adds your article to Technorati's tag index when you ping their servers with the URI of the article.
Further, imagine POSTing to some new site a link to your hCard on your own server, to save you having to type your name and address again. And you'd never need to manually update sites when your address changes: just ping 'em all.
eA: Ah - but I thought all URIs should be GETable. The ping URI you're POSTing to in these examples isn't always one that you can also GET!
DC: Indeed - so think how much more powerful it would be if we did close the loop and provide or create a GETable resource to POST these notifications to.
For example, imagine a page containing an hCalendar event. Now point to it with a rel="attending" link. When the hCalendar discovers your intention (using a direct POST ping of your page's URI to the hCalendar page's URI - or perhaps through the referrer trick from people clicking through), it adds your referring page to a list of attendees inside the hCalendar. The hCalendar could either contain lists of backlinks to the attendee's pages, which may in turn carry hCards, or it could contain lists of complete hCards copied over.
eA: Sounds like a good use of Microformats.
DC: These examples make crawling and polling (even with If-Modified-Since et al) look like a clumsy version of the more proactive POST.
Web Feeds and general publish-subscribe are further examples where POST may be used to notify changes on a resource - giving the feed consumer first-class resource status with their own URI.
eA: I'd never think of using HTTP in this way.
DC: Obviously this only applies where the feed consumer is a visible and POSTable server and where timeliness is crucial. And probably where the number of subscribers is relatively small, unless asynchronous I/O and an event-driven architecture are employed, and you don't wait for the response to each POST.
This isn't done now simply because of the asymmetry of the current Web, an asymmetry which we are free of in REST integration.
eA: What about all those REST rules about idempotent and unsafe methods?
DC: We're not mixing GET and POST in that sense, just turning the tables on the asymmetric Web. GET is still cacheable, and we can POST a link to cause a cached GET.
I believe this is a more-constrained REST style, not disjoint to REST. It is at least an ROA! It may fall foul of REST's client-server constraint, since we're now in server-server territory with integration applications. Also, the concept of 'Hypertext as the Engine of Application State' is something that may take some refitting to the mutual state dependency model. However, I believe it's most important to focus on maintaining the benefits of REST and its key elements of standard content types at URIs.
I call this symmetric REST integration style the 'Distributed Observer Pattern'.
eA: Quickly summarise the 'Distributed Observer Pattern'.
DC: OK, the Distributed Observer Pattern is 'symmetric REST'. A resource subscribes to a peer resource via a GET that supplies its own URI, and is notified of subsequent state changes in that resource through a POST back.
eA: That was too quick. Tell me the details!
DC: OK, here are four. First, a POST can be either the whole new state or the fact of the change, allowing the subscriber to GET the resource when it's ready (and thereby fill any caches).
Secondly, you can use either the Referer header or perhaps the Content-Location header in POST and GET requests to indicate the origin POSTer or GETter URI. Alternatively, you can send this origin resource URI using the Cookie header, echoing its use in the normal browser client-server case to identify the pseudo-resource of a browser user.
POSTed state notifications may be unsolicited by a prior GET subscription, when the POST target is clearly open to them (as in the ping notification examples). These can be seen as 'subscribe to anyone', and may be combined with a corresponding 'GET anyone' crawling process, without explicit subscription.
Finally, POST notifications may be targetted to single resources to ask them to update: the Distributed Observer Pattern way of achieving the client-server editing function. These now become 'edit suggestions' of the POSTer resource - putting the target back in control of its own destiny and integrity.
eA: And why should I use the Distributed Observer Pattern?
DC: The Distributed Observer Pattern supports the programming model of inter-dependent resources whose own state is a function of their peers' state, driven by declarative rules. It's a very general ROA programming model.
(c) 2006-2007 Duncan Cragg
In Part 6: Content-Types and URIs.
Note that the opinions of our imaginary eBay Architect don't necessarily represent or reflect in any way the official opinions of eBay or the opinions of anyone at eBay.
Indeed, I can't guarantee that the opinions of our real blogger necessarily represent or reflect in any way the official opinions of Roy Fielding...