Don't write your interactive Web application in custom Javascript! The Web's Declarative nature needn't be broken just because you want two-way dynamic data instead of one-way documents on your site.
Instead, write Declaratively to generic Javascripts, plugins and browser features such as Hijax, hInclude, XForms, SVG, XBL, etc. ...
Don't write your interactive Web application in custom Javascript! The Web's Declarative nature needn't be broken just because you want two-way dynamic data instead of one-way documents on your site.
Instead, write Declaratively to generic Javascripts, plugins and browser features such as Hijax, hInclude, XForms, SVG, XBL, etc.
This is the last of my XTech 2006-inspired articles. (I know: XTech was some weeks ago now, but I'm sure you'll have noticed that this is a 'contemporary article' blog, not a news blog!)
The theme for XTech 2006 was 'Building Web 2.0', and as a Web 2.0 Conference (don't tell O'Reilly), there was tons of Ajax... And as this is essentially an XML Conference, there was also a strong Declarative theme.
And the right way to do Ajax is Declaratively!
Simon Willison presided over this session. The demo that impressed me most was Hijax: Progressive Enhancement with Ajax, whose paper is here.
Hijax takes the view that you should design for no Ajax, then hijack the links, form posts and page construction in Ajax to speed it up, rather than designing for Ajax in the first place and excluding users without a capable browser. This is related to the AHAH idea.
So you don't build your application in Javascript, but simply enhance the existing Web model, running all the page's behaviour back on the server which owns it. Much more Declarative!
Mark Nottingham is one of those people whose blogs are top of my feed list. He currently works for Yahoo! This talk was about using some basic techniques to ensure Web 2.0 doesn't grind to a halt with all those dynamic and personalised pages. Simple fixes can get you very far: faster servers like lighttpd, various cacheing tricks, etc. Yahoo! should be implementing his ideas and insights on their sites (hopefully Mark will then go on to tell their API designers how to do REST...)
The main idea that interested me was using Ajax for client-side page construction (think client-side SiteMesh) and for inserting personalised URLs into the page.
It's a bit like Hijax, that I mentioned above. Mark says:
What I would like to see is for common JS functions like this to be sifted out and standardised as declarative markup...
This is good stuff (assuming you agree with us that Declarative is good!). See more in the summary and slides, and in Mark's hInclude proposal.
Ditching the database: XML and the PHP webapp
David Megginson basically said we should work in XML throughout our web apps: cache XML as your datastore, keeping all the little snippets that make up your pages - dynamic and static - cached in memory, then just send XML snippets from the server cache to the Ajax client to render, etc. This is rather like Mark Nottingham's tip above on client-side page assembly to achieve greater cacheability.
Introduction to XHTML2 and XForms
The flamboyant-looking and entertaining Steven Pemberton of the W3C presented a fast-paced and in-depth overview of XHTML2 and XForms.
Unless I missed something, it looks to me like XHTML2 has taken a perfect trajectory between backwards-compatible concepts, semantics and syntax and abstract idealism. XHTML2 appears to cleanly fix all the outstanding problems with HTML once and for all.
Now, part of XHTML2 is XForms. XForms does the same cleaning up job on HTML forms that XHTML2 does on HTML.
The important thing to know is that it works as an XML 'schema constraints engine'. That is, it returns to the server a fully-semantically-valid chunk of XML on a form submission. The 'schema' can include some static data pulled in from a separate URL to populate drop-downs, etc.
Yet it is more than this: I asked if it was Turing Complete, and Steven said yes. You can program an application using it.
And that application will be programmed Declaratively!
I can define the constraint that some output element be dependent on some input, then if the user changes the input, the output changes accordingly.
It's like Declarative Ajax. There were further talks on XForms which I cover just below.
Of course, I asked if it was RESTful, and it seems it was, mostly. The issue was that a URL may bring you in an editor XForms page, which then itself brings in some data underneath.
The editor XForms page is the bookmarkable thing but you don't see the URL of the populating data, so that's not bookmarkable. The page may change according to this data, but bookmarking what you see still only bookmarks the editor, not the editor plus the current visible data.
Also, even though XForms is amazingly programmable, it is still a forms system at heart. For example, you have explicit input and output elements; you can't get input events off the DOM and rewrite the DOM as output.
Slides here.
Read more about XHTML2 here.
Read more on XForms from Mark Birbeck (a Pemberton collaborator) starting here.
Building Rich, Encapsulated Widgets Using XBL, XForms and SVG
Mark Birbeck showed us how to create a Declarative client-side mash of Flickr. OK, there's a tiny bit of script in there, but I'm sure it's unnecessary, as it only does some simple DOM manipulation and event handling. Perhaps it can be replaced somehow, or made part of a generic library.
XForms: an alternative to Ajax?
Since XForms is so amazingly programmable, you can use it instead of Ajax.
Instead of coding a special script on top of an Ajax library, you can bring in a generic (Ajax) script that gives you XForms today, and write your application Declaratively on top of that.
Alternatively, you can do some pre-processing on the server, which is the approach Erik Bruchez explained here.
The Power of Declarative Thinking
This talk was more from Steven Pemberton giving us a little history of his work and showing us how XForms can be used for Declarative programming.
Some good quotes from the slides:
.. a program that is 10 times longer is 32 times harder to write.
Or put another way: a program that is 10 times smaller needs only 3% of the effort.
And of course, the point is that Declarative programs are usually 10 times smaller.
.. no one writes applications except programmers.
Interesting exception: spreadsheets
Mostly because they use a declarative programming model.
The nice part about declarative programming is that the computer takes care of all the boring fiddly detail.
Now I'm as aware as anyone of the dangers of giving non-technical people the power to program. Spreadsheets are, let's say, a mixed blessing. There's probably a long way to go before we can give the business the reins over their own business rules.
But it will happen: it means splitting programmers into (Declarative) Business Analysts and (Imperative) Those That Enhance and Maintain The Underlying Processing Fabric. I can tell you now which group will get paid more.
Standardising Web Applications: Rich Web Clients at W3C
Dean Jackson gave a talk about what the W3C are doing to play catch-up with all the latest goings-on in the Wide World of Web 2.0.
Here's a quick list of things they're standardising: XMLHttpRequest, Networking, Timers, Client-Side Storage, File Upload, the Window class, DOM Level 3 Events and XPath, Drag'n'Drop, Remote Events for XML (REX); XBL2, XUL (possibly alongside XAML, MXML and LZX); Compound Document Formats.
Again, the magic word Declarative came up in the talk.
The Declarative aspects of Rich Web Clients centre around the DOM Level 3 work, Drag'n'Drop, Remote Events for XML (REX), XBL2 and XUL and Compound Document Formats. SVG is a common theme between many of these (once again, Sam Ruby is on the case).
The WHATWG is an independent collaboration of browser builders (Opera, Mozilla, Apple) that has its own initiatives in this space - including HTML 5 and Web Forms 2.0, the Canvas object, server-sent DOM events, network connections, client-side state, drag'n'drop, etc. They are happy to work with the W3C to ensure compatibility and W3C standardisation. Here is an article covering some of these WHATWG ideas.
Other Declarative Browser-Driving Work
When you look around outside of the Conference, you suddenly discover numerous further examples of Declarative browser-driving:
Tim Bray has been stitching pages together with Ajax.
Sam Ruby says Declarative is more mashable.
AHAH is 'Asychronous HTML and HTTP', an apparent attempt to make a big deal out of rewriting the DOM on receipt of an Ajax HTML response. Same ideas as above.
Jemplate looks like a similar kind of thing.
XAP (Extensible Ajax Platform) is an interesting project that was given by Nexaweb Technologies to Apache. Here's the elevator pitch:
Widgets intersect with this whole dynamic, client-side page-assembly space. There are now numerous examples of commercial Widget approaches. Some Widgets may be driven by a Web Feed, some may get their data from a Microformat or from a Web API.
Opera Widgets are DHTML packages that break free from the browser and run on the desktop. The WHATWG could even end up taking them as the basis of their play in this game.
Of course, we've already seen how to do Widgets right, in Mark Birbeck's talk above. In general, Compound Documents and dynamic, POSTable Microformats are the (Declarative) future of Widgets...
Declarative Meme
We need the 'Declarative' meme to consolidate and create a movement out of all this, otherwise apparently disparate, work. Once we have a meme to gather around, we gain collaborative and cognitive power from its mere existence. This could work in the same way that the label 'Ajax' itself consolidated (and launched into orbit) techniques that a large number of us had already been doing for months or years.
The 'Declarative' meme also includes and extrapolates the 'REST' meme. REST is Declarative, but not something you may normally associate with Ajax application programming. Yet programming a dynamic Web interface Declaratively goes hand-in-hand with the linkability, cacheability and mashability of a RESTful back-end interaction, as Mark Nottingham and others above have shown.
The Right Way to do Web Applications is Declaratively
Specialist client-side Javascripts are a mistake. They break the generic Web client model: hand-crafted thick clients are hard to make cross-browser compatible, they add to download times, they are tied to specific application servers, they break the browsing experience (bookmarking, linking, history) and miss the benefits of the Web's cacheing architecture.
We should be writing Declarative pages to exploit generic Javascripts and plugins or extensions (such as SVG, XForms and other Declarative technologies). These generic facilities will effectively extend the concept of the browser, from one that deals primarily with one-way static documents, to one that deals with two-way dynamic data.
After some time, the browsers themselves will be written to new standards that make those browser-redefining Javascripts and plugins irrelevant.
By a 'two-way dynamic data' browser, I mean a browser that constructs pages from several sources - several snippets, microcontents and widgets. It will show those pages updating in real-time as those sources update.
It will be aware of the user's key and mouse events on the visible DOM, and relay them back to the affected source, which may in turn update - for everyone viewing the page.
Programming the behaviour behind the data can be completely Declarative; rule-driven and event-driven. The programs will run both on the server, where the Resources are owned, as well as on the client, as in XForms.
I've shown many examples in this article that are heading in exactly this direction.
New Web (2.0) standards, that respect and extend Web architecture, can one day replace hand-crafted thick client scripts and generic scripts and browser extensions, allowing us to program our dynamic applications Declaratively - directly to a Web 2.0 browser.
Such a Web 2.0 architecture and platform will still be as scalable and interoperable as the current Web architecture - more so, in fact. And it will be much more fun to program for!
Open Data .. has .. recently .. been.. all .. over .. the .. blog .. o .. sphere!
Openness is a classic Us-and-Them issue. Big, nasty Apple/MySpace/Flickr is trying to control what little me/SingleStatus/Zoomr can do with my/our own stuff.
Open Data vs. Closed; Open Source vs. Proprietary; P2P vs. DRM; privacy vs. surveillance. The battles between the freedom of the pioneer, the individual and the minority against the rules and stability of the establishment and the majority form the endless shape of human history.
Us beating Them is Hollywood's favourite subject on-screen - and ironically Them fighting Us Hollywood's favourite battle off-screen.
As an Us-and-Them issue, with Us less powerful than Them, it's also tempting to give up and to follow the crowd - to do what we're told, to not ask for or sieze the privacy and open data we feel entitled to.
However, at XTech 2006 recently, there was a set of talks on the subject with a more positive approach. ...
Open Data .. has .. recently .. been.. all .. over .. the .. blog .. o .. sphere!
Openness is a classic Us-and-Them issue. Big, nasty Apple/MySpace/Flickr is trying to control what little me/SingleStatus/Zoomr can do with my/our own stuff.
Open Data vs. Closed; Open Source vs. Proprietary; P2P vs. DRM; privacy vs. surveillance. The battles between the freedom of the pioneer, the individual and the minority against the rules and stability of the establishment and the majority form the endless shape of human history.
Us beating Them is Hollywood's favourite subject on-screen - and ironically Them fighting Us Hollywood's favourite battle off-screen.
As an Us-and-Them issue, with Us less powerful than Them, it's also tempting to give up and to follow the crowd - to do what we're told, to not ask for or sieze the privacy and open data we feel entitled to.
However, at XTech 2006 recently, there was a set of talks on the subject with a more positive approach.
I'm counted amongst the 'founding members' of the UK's Open Rights Group, having signed the pledge in PledgeBank. So I was delighted to see Suw Charman's name on the XTech programme.
Suw gave a talk about the issues they/we are tackling: ISPs and Telcos tracking our traffic, DRM controlling our media and even our computers, ID cards, Patents, various government schemes to undermine our privacy, Copyright extension, Trusted Computing, etc., etc.
Read about the issues, then join the group (or the EFF, etc., depending on where you live).
Now, this was cool. I'll start right off by noting that they have a REST API, not a STREST API..
And it just gets better from there.. OpenStreetMap is like a Geo Wiki of GPS trails around the world. The centre of London was mapped by attaching GPS devices to couriers and plotting the slug-trails they left behind. Steve Coast showed an animation of a day's tracking. The trails shot off to various points out of town at the end of the day. But the animation continued..
And at pub closing time, another burst of trails as the couriers came home!
A few weeks ago, a crowd of OpenStreetMap volunteers descended on the Isle of Wight in England to walk around mapping it. The enthusiasm generated by this project seems likely to give it enough momentum that it will very soon be good enough for us to use for real. A complete, reliable dataset created by the people, for the people, unencumbered by copyright and restrictive licensing.
I suggested an idea I had about five years ago in my dot-com days: let people upload mobile-snapped pictures with GPS coordinates attached. Central servers scan the pictures for text and OCR them into a searchable database.
People would snap-and-upload their own street name signs, house name plaques, shop fronts, station name signs, company office signs, etc. Even menus in the local restaurant. We could construct a 'GeoGoogle' of public, mappable, searchable text.
However, Steve told me that OCR and mobile GPS technology still isn't up to it; that manual tagging of GPS trails is enough to make OpenStreetMap work well. Ah well, I've waited five years, so I can wait a bit longer...
Paul Hammond (ex-BBC, Yahoo!) gave a good talk on how to motivate companies to open up their data to the Mashable Data Web, hopefully through nice shiny new Web 2.0 APIs.
Here's your takeaway quote courtesy of Suw Charman's amazing typing:
- Be aware of the problems
- Demonstrate usefulness, screen scrape if you need to, but don't get yourself cease-and-desisted
- Don't assume it's a technology problem
- Target the right people, find someone on the inside who can help you
- Talk about benefits to the provider, not the consumer. If you talk about the benefits to you, they'll see you just as someone who wants something for free.
- Have patience. It is getting better every day, and it takes time for business to come round.
The Data Web will be created mostly from the edges, but the middle should be encouraged to see the benefits, too.
Native to a Web of Data: Designing a Part of the Aggregate Web
Yahoo! had a big presence at XTech - as did the BBC - and this Yahoo! - and ex-BBC - presenter was Tom Coates.
Tom's talk extended the previous one of his colleague, Paul Hammond, by going over some principles of opening data and helping build the Data Web.
Principles such as finding data sources, giving them nice representations and URLs then distributing them in a most accessible and navigable way.
A bit Web 1.0, in truth; he looked rather blankly at me when I suggested adding some concept of data events or updates to his list of data-opening principles.
Again, Suw has this written up (actually, a better-looking version of the same talk from FoWA).
Oh - here's a good article by Tom on the power of pointability
Harnessing the Collective
All of these talks had a positive spin on the Us-and-Them issue.
We can do something about the creeping imbalance of Digital Rights, tipping slowly but persistently from Us to Them.
Web 2.0 and the Social Software it underpins can actually empower those of us that are willing to use it.
As taxpayers, voters, shareholders, pensionholders, customers and employees, we really do own and control Them. We are just one Social Networking movement away from realising that - in both senses of the word.
So: stop staring at YouTube, sign up to your local Digital Rights organisation, get out on your bike with your GPS, understand and talk nicely to that unenlightened, data hoarding corporation and start a bottom-up revolution to take back control over your own data...
Above all, design your systems to put Open Data at the top of your list of priorities...
Microformats are subversive: they not only challenge the approach of full-blown Semantic Web approaches, but even question fundamental Web 2.0 building blocks such as Web Feeds and Web APIs.
I recently attended XTech 2006, where there were a few talks related to Microformats.
After summarising these talks, I'll finish with my shocking revelations about the subversive nature of Microformats! ...
Microformats are subversive: they not only challenge the approach of full-blown Semantic Web approaches, but even question fundamental Web 2.0 building blocks such as Web Feeds and Web APIs.
I recently attended XTech 2006, where there were a few talks related to Microformats.
After summarising these talks, I'll finish with my shocking revelations about the subversive nature of Microformats!
Microformats from the Ground Up
Ryan King and Brian Suda from Technorati presented a half-day tutorial.
You can describe social networks, calendars and various other semantic relationships by annotating page elements or embedding small XML structures into your page according to a Microformat spec.
Microformats take Declarative or semantic markup (e.g. in XHTML) to the next logical level. Microformats have been called the 'lower case Semantic Web', because they fill the need for a lightweight set of conventions for data in Web pages that transcends the merely renderable.
HTML has a number of extension points that make this kind of thing easier. For example, the 'rel' and 'rev' attributes on a link (say what type of link it is - maybe a tag); the 'class' attributes (used for CSS but can also carry add-on semantics); the 'profile' element (say what kind of thing the whole document represents).
Microformats examples include rel-tag (for tagging your blog entries in Technorati, etc), hCard (like vCard-on-a-page), hCalendar (like iCalendar-on-a-page), XOXO (outlines) and XFN (social networking). Technorati have an hCalendar subscription service.
However, the one that stood out for me was hAtom, with which you can publish a blog and then let feed readers poll the actual page. They can read the hAtom markup and treat it as an Atom feed.
The Intelligent Design of Microformats
Ryan King also presented a talk on the motivation and style of the Microformats effort.
There is no official standards body defining Microformats, just an "Open Source"-style engineering effort, which follows the following list of Worthy Engineering Principles: "Rough Consensus and Working Code", "Paving the Cowpath", "Keep It Simple and Stupid", "You Ain't Gonna Need It' and "Don't Repeat Yourself".
Microsummaries in Firefox and on the Web
Myk Melez gave a presentation about Microsummaries in the up-coming Firefox releases.
To quote:
Microsummaries are regularly-updated compilations of the most important and timely information on web pages. For example:
- current stock price and movement for a company profile: "GOOG: 406.74 + 0.58"
- latest headline for a news site: "BBC: US dismisses Iran attack claims"
- highest bid and time remaining for an auction item: "Godzilla DVD: $15 / 2 minutes left"
One way to create this information is to link ('rel="microsummary"') from a page to its associated Microsummary document. You can also use XSLT to build the summary on the client. Both approaches struck me as clumsy.
I commented that it would be more Microformat-like to allow embedded markup (just annotate the Microsummary information right in the page; perhaps call it 'hSummary'?).
Myk used the bandwidth and page generation cost argument against that, but good XHTML pages should be very lightweight, and there are various AJAX techniques that can assemble heavier pages.
The current manifestation of the idea is to write this summary on a corresponding bookmark button in the chrome.
Allowing just bookmark buttons seems rather restricted to me. I'd be happy with a whole page in its own tab (or a widget in an Ajax desktop).
A drag-and-drop prototype was also shown, where you visually mark up the parts of the source page you want summarised. I commented that a good extension of that would be dragging across elements to a whole new page, whose construction would be dependent on the information in one or more source pages.
Myk didn't jump up and down with joy at my suggestions, so I guess we'll have to wait before we get Microsummaries written into Web pages that generate more Web pages.
RDFa: The Easy Way to Publish Your Metadata
Of course, the Semantic Web folk have their own Microformat approaches, including this one.
To quote from their paper:
The approach taken by RDFa is that ultimately any RDF structure should be representable. This means that instead of having to 'codify' each format to describe how it must be marked up, we simply provide a set of rules that explain how any RDF can be marked up, and than any RDF 'language' can be used.
The 'a' stands for 'attribute': RDF encoded as attributes in XML.
The example shown was FoaF, whose Microformat competition is
XFN and hCard.
You can use attributes like <link rel="foaf:mbox">,
<span property="foaf:name">, <span property="foaf:phone">
to encode RDF triple information at various points around your
document.
RDFa has some arguable benefits over their competition, but I don't see why both sides shouldn't get along and share ideas. And, indeed, why the other alternatives (structured blogging, embedded RDF) shouldn't all find a place, too. There is a comparison here. Another speaker at XTech, Uche Ogbuji, has also discussed some of these issues.
Web 2.0 Centraal
Microformats not only challenge the Semantic Web, they even challenge such basic Web 2.0 technologies as Web feeds and Web APIs.
Shouldn't we be subscribing to and interacting with semantic Web pages (note the lower case 's'!), not using invisible Web Feeds and Web APIs? Technorati think so: they are leading the way in searching and subscribing to Microformats.
The redundancy of Atom itself (and feeds in general) has been remarked upon before. Further, since we're using REST in the Atom Publishing Protocol, why not use the same protocol to publish directly to an hAtom annotated site? (e.g., just POST a new article to your own blog's main page!) Web 'API's are redundant when our pages are just XML data anyway.
Writing data pages instead of document pages, then allowing those pages to be updated and then subscribed to, is the future of the Web: it's Web 2.0 Centraal.
The new lightweight, Declarative, Microformat-rich XHTML pages we're going to be generating can be polled just as Web feeds are, and events made from their changes. Then re-presented via Ajax in dynamic pages. Such standardised page formats and micro-formats should also accept standardised POSTs where appropriate.
Microformats represent some first steps towards the Data Web replacing the Document Web. Although there has been some thought given to extracting full-blown Semantic Web data from Microformats, there's a significant chance that this bottom-up, simple, minimal, change-aware (i.e., updateable, subscribable) approach may teach the Semantic Web practitioners a thing or two about the Worthy Engineering Principles that enable the social and technical proliferation of a good idea.
Standardising common formats and structures is a precursor to this two-way, dynamic Data Web. Both the bottom-up practitioners of Microformats and the top-down practitioners of the Semantic Web have much to learn from each other on this path.