eBay Architect: OK, enough fancy REST or ROA interaction patterns! Let's get back to REST basics.

Duncan Cragg: That'll be content types and URIs, then.

eA: OK - I've a question about your standard content types. In particular, about Microformats.

DC: Go on.

eA: You keep on mentioning Microformats when offering examples of content type standardisation. But Microformats are always embedded into HTML (or, preferably XHTML).

Are you suggesting somehow taking them out and using them separately? Or do we have to always use them through a, possibly inappropriate, document model schema?

DC: Microformats currently 'tunnel' through mechanisms in XHTML such as the class attribute. They themselves have distinct model schemas, that ride on the document model schema.

eA: So can you use them apart from HTML?

DC: No, not really, although you can carry a single Microformat in a single document carrier, then squeeze out any document model content that isn't supporting the Microformat, perhaps leaving enough that it still renders sensibly in a browser if anyone looks.

The Microformats people won't like you doing it, I'm quite sure, but if you and I want to exchange a pure hCard, hCalendar, hResume or hReview, and nothing else, then we can use the minimal document model carrier, and have just one Microformat per resource.

eA: But why not use the original data schema, before it was Microformatted? Why not just use vCard and vCalendar?

DC: Or use Atom instead of hAtom! Of course, if vCard has an XML representation, you could use that - as long as the constituency of your clients is the right one and is big enough. There may be more code out there that 'gets' hCard than an XML vCard. And some Microformats - such as hResume and hReview - don't have an original schema and are based on abstracting from common or prior behaviour.

This is REST integration we're talking about, where data, not documents, are native, and we aim to search out the most popular and most widely understood data schemas - even if carried over documents - to maximise interoperability.

eA: OK, that seems fine. Although I'd point out that REST doesn't have the monopoly on interoperability. SOA does that too.

DC: Interoperability is best acheived by sharing millions of URIs dereferencing to a handful of standard content types, with interlinks across the Web of resources. ROAs (Resource-Oriented Architectures) do that. SOA doesn't.

eA: REST APIs don't always have to do it. In the previous example you went through, eBay and gBay could offer REST interfaces but not talk the same schemas and not allow cross-linking in the way you described. Or talk the same schemas but not recognise each other's Items and Offers.

DC: That would be walled-garden, silo-thinking. It's also 'API' thinking. Just opening up a port to your application, even one with correct use of GET and POST on well-organised domain URIs, isn't in the spirit of REST, and certainly isn't good enough for REST integration.

In REST we always aim to adopt the same schemas, to aim explicitly for interoperability. And linking between those resources, even cross-site, is fundamental to the REST way of thinking. If someone offers you a 'REST API' that uses unnecessary proprietary schemas that miss obvious interlinking opportunities, especially across to other sites, run away!

eA: Are there any real-world examples?

DC: A good, and ironic, example of this is Google's Open Social, at least in its earlier releases, which fails to achieve true cross-site openness even with a 'REST API' and shared schemas, because sites don't cross-link or actually allow data sharing. Also, the schemas are a strange extension of Atom, rather than using, for example, vCard as the basis for 'People Data'.

This hopefully will be fixed as the 'REST API' evolves and with the work going on in groups such as DataPortability, with agreement from the major operators.

eA: So much for the interoperability of that REST interface.

DC: The heart of good REST interoperability is the acceptance of standardised data at a 'foreign' URI, and the re-publishing of that foreign URI in your own standardised resources. It happens on the Web all the time, of course. We just need to copy the model for REST integration.

Hypermedia and (more importantly, here) 'hyperdata' is baked into REST, but is an afterthought in SOA. ROAs create an interlinked hyperdata landscape across sites and domains. I'm using 'hyperdata' here in the sense of interlinked data resources in REST integration, by analogy with hypermedia, not in its Semantic Web sense.

eA: Ah! But how do your little pure Microformat resources link up into this hyperdata landscape? Microformats can't link to each other, can they?

DC: It's true you may have to go and get involved in the Microformats movement in order to help define how to link an hCalendar event to a list of hCards of people attending. Or the hCard of a company to a list of hCards of its board members. Or an hReview to the hCalendar event being reviewed and the hCard of its author. Or to include the XFN list of links to friends' hCards inside a person's own hCard.

One indication that there's something not ideal in Microformats is the fact that you have to write someone's hCard out again and again for every page or site they appear on. If you could just link to a single hCard for that person it would be more efficient.

eA: But Microformats have a narrow charter: to decorate the document model with semantics. Any links are just part of the hypertext Web. It sounds like you're trying to make some kind of domain model out of them, with their own interlinks!

DC: Yup. When you start to think the data of REST integration, the document carrier of Microformats and it's often superfluous links can be a distraction. If the document links are relevant to the Microformat, of if people would use links within the Microformat if they were told what value it has, it would be worth pulling them out into the Microformat definition itself. Then enhancing in-browser Microformat parsers to follow links will greatly enhance their utility.

All you have to do is find real-world examples, and propose it on the Microformat lists! Meantime, reuse the schemas and keep all your extensions public and backwards-compatible.

eA: What about all those 'rel-' decorations? You know, rel-tag, XFN, etc.

DC: Well, hAtom is the only Microformat that specifies nested rel-links: rel-tag, rel-bookmark and rel-enclosure. Otherwise, each Microformat is independent, and the rel-links are independent. Like I said, it may be worth going to the Microformat community and suggesting more such rel-links beyond hAtom.

URI Opacity

eA: So this RESTful data landscape of data wired up with URIs: it sounds a bit hard-wired: where do URIs as queries (and URI templates) fit into that tight mesh?

DC: URI templates fall into exactly the same category as standardised content types and schemas in terms of their level of abstraction and location in the stack. In other words, the right thing to do, if it's transparent URIs you want, is to standardise search URI templates across sites of a type.

eA: This is getting complicated. It's hard enough to get agreement on the content types of resources, never mind on URI formats as well!

DC: Indeed, and in fact, I believe that URIs should be opaque: they already are to HTTP, but also in our data landscape, a URI should point to a single, predictable resource.

The mechanism of querying that dataspace should be separated out from the mechanism of linking it up.

eA: A bit like GUIDs?

DC: Exactly. In Enterprise applications, you often see GUIDs (globally unique ids) being used, and never see them mixed up with search strings!

Transparent, query or template URIs are either used to be helpful or decorative, or are an acceptable optimisation, as long as you know that it's tunnelling through or hijacking the URI for a quick query string.

eA: Tunnelling? Hijacking? You've dismissed a long-standing convention, in the Web at least! How else do you do query fetches?

DC: A better solution is the query-POST-redirect pattern: the client POSTs their query, then the server redirects them to a linkable results resource on an opaque URI.

The POST query schema can then be properly standardised in a content type, or 'templated' in the REST integration equivalent of an HTML form.

It's an extra round trip, but only one IP packet in each direction; a redirect or a GET can fit into a single IP packet - the cost is only in the connection latency.

eA: Why not just return the state of the resource you're redirecting to in the body of the redirect, to save even this round-trip?

DC: Yes, you could do that. It's not something seen in the hypermedia Web as far as I know, but this is REST integration, where we're able to come up with new sub-protocols like this - where HTTP response codes are often given much thought.

Further, the server can offer the option to snapshot this results resource, so that it's still exactly the same whenever the link is dereferenced - something you can't do with a query URI.

eA: What would Tim Berners-Lee say about this? Is it in the spirit or letter of his vision for how HTTP and URIs should be used?

DC: I've no idea! However, in my opinion, when Tim didn't separate the concepts of a globally unique identifier returning exactly one resource from a query string returning maybe none, one or many resources (in a list), he started a good deal of unnecessary confusion, even if non-fatal in practical terms.

The phrase 'hackable URIs' sums up the situation. We may have been forced into creating slightly better user interfaces if the URI textbox were taken away from browsers.

Make your interface your content and have good search and information architecture to allow your (opaque) links to be discovered. If you know that human users - or search engines - will be interested in reading some links at the top of your information architecture, then go ahead and use just a few simple, meaningful addresses.

eA: You're venturing into controversy again! I'm sure I keep reading about designing nice URLs being good practice.

DC: There was a time when transparent URLs were considered important, but now everyone just uses Google! All the energy that's put into URL good manners and systems of URI templating and naming is just a distraction from the bigger effort of standardising content and defining schemas.

Opaque URIs keep content in the body where it can be given a Content-Type, instead of the headers - the URL line.

This is related to my preference to put 'write methods' such as PUT and DELETE into the body instead of the URL line.

eA: How exactly?

DC: The URL line should have a definite target - an opaque, globally-unique URI - and a content transfer direction - GET or POST.

The rest of the application-level interaction, including anything that will affect state and any searching and querying, should be in transferred bodies with standardised content types.

(c) 2006-2008 Duncan Cragg

In Part 7: Business Conversations

Note that the opinions of our imaginary eBay Architect don't necessarily represent or reflect in any way the official opinions of eBay or the opinions of anyone at eBay.

Indeed, I can't guarantee that the opinions of our real blogger necessarily represent or reflect in any way the official opinions of Roy Fielding...