What Not How - Posts tagged 'web2.0'

Mobile Widgets aren't the Mobile Web

2009-02-11T16:20:00Z

Mobile Monday London met last night to discuss the Mobile Web and Widgets. It was an engaging and thought-provoking evening.

Your intrepid reporter was there and, in spite of the crashing of his sad, clunky old Windows Mobile Xperia X1, losing all his notes, he brings you this hot report from right out of his memory (somewhat steamed up by subsequent socialising, but reclarified by Google).

After that, I give an explanation of why I believe that Widgets are not the solution to what Mobile 2.0 needs...

GSMA ONE

First up was Kevin Smith from Vodafone to tell us about the GSMA ONE Web API (also via here). ONE means 'Open Network Enablers'. It seems quite similar to the Web21C initiative by BT, led by my good friend Paul Downey, but which was unfortunately set aside in favour of Ribbit.

You can send an SMS, get a user's location and access billing and connection info. All through a scary STREST interface.

Scary because it looks like you can send a text using a GET... The documentation here [PDF] does suggest using POST, but even so, the design fundamentally wants Resource-Orientation in place of Service Orientation:

The message appears to be tacked on to the URL as an argument. There are references to 'soapUI', endpoints, etc. An SMS message doesn't appear to have its own URL, just an internal message ID. There are 'exceptions thrown', all with a 400 code, even for internal platform and integration faults. And so-on - in classic STREST style!

Still, nothing that can't be fixed with a little help from a REST dude, or a read of "RESTful Web Services".

OMTP BONDI and Ikivo

Next, Nick Allot told us all about OMTP BONDI, which is an attempt to ameliorate fragmentation in the mobile Web and widget space - the one which allows access to parts of the phone not normally reached in a browser: PIM, SMS, camera, GPS, etc.

Their big thing is security: take widgets out of the safety of the browser and give them access to the phone via APIs, and you have a potential security nightmare.

Seems BONDI will let you do most Mobile 2.0 native-like applets, such as LBS, photo-sharing, etc. There's a reference implementation for sad, clunky old Windows Mobile, and both Opera and LiMo are jumping aboard, too.

BONDI aims to be W3C compatible where possible. Not HTML5, mind, but rather a works-now, fit-for-purpose solution to the fragmentation problem, that includes widgets and may later converge with HTML5. There's an excellent discussion about this on WAP Review.

This was followed by a talk by Samuel Sweet from Ikivo. It seems that, starting with their successful Samsung T*Omnia widget interface, Ikivo have been overcome with standards love, and are going BONDI as well as W3C compliant, especially with SVG as a rendering technology.

Firefox for Mobile

Christian Sejersen of Mozilla gave a good intro to what is currently called Fennec - Alpha 2, but will soon be renamed the more grown-up 'Firefox'. It's got the same code in it after all: and APIs such as camera and location that get added to the common codebase will be accessible via both mobile and desktop.

Looks very swishy and touchy and in sync with the times. Available now on Maemo (and Moblin) and soon on sad, clunky old Windows Mobile. Then later this year on Symbian. Codebase is C, so no Java or Javascript phones supported, of course. Or closed walled-garden proprietary locked-down phones like, um, the iPhone. It seems that the add-on community is already fixing up their plugins for this mobile version without even being prompted. No news on the Prism widget-alike system, to compete with BONDI or Opera. It's "still in the labs"...

Panel

Then the panel session started, with the best questions from the excellent host, Dan Appelquist of Vodafone, and some good ones from the audience.

There was much elaboration and clarification on the presentations (thus included above instead), and an interesting conversation around monetisation: how do widgets make money? Three suggestions: adverts, selling widgets on an app(let) store and micro-payments. Graham Thomas of T-Mobile appeared to be happy just to get more Internet traffic for now..

There was also confirmation from Graham that Web'n'Walk 4.0 will major on widgets, but he didn't say what flavour. More hot journalism uncovers this confirmation [PDF] (page 29) that it's still going to be tied in with Opera.

The outstanding revelation of the evening (sorry Ikivo!) came when someone told us that the Palm Pre's widget system is not only proprietary, but uses tables for layout! The horror! Lots of tutting and W3C-like smugness around the room.

Following Open W3C Standards can still break the Web

The goal of all this standard widget aspiration, apart from the obvious motivation of being able to compete with the iPhone, is to allow everyone to write widgets, and for those widgets to work on all our phones.

However, just because everyone does what the W3C thinks they should do doesn't mean you automatically get interoperability. Most importantly, doing what the W3C wants doesn't mean you won't break the Web!

The W3C, don't forget, also brings you 'Web' Services - those mis-named standards responsible for much Web-breaking, including inspiring a whole generation of Web-breakers with their STREST interfaces.

Interoperability on the Web is about state transfer, content types, URLs, hypertext. No amount of API and widget standardisation will give classic Web interoperability as long as they ignore these Web basics.

A widget is an applet is an application. It's a closed structure whose data interoperability has to be hard coded each time, and whose imperative Javascript naturally wants to make function calls back on the server - probably through HTTP.

And you run either this application or that, each having its own way of pushing or pulling data around, or even the same data presented in much the same way all over again.

In the Web you run one browser application and everything is mashed within - data interoperability.

The U-Web gives us the best of both worlds

Admittedly, the basic Web isn't good enough by itself when seeking an architecture for Mobile 2.0's essential personalisation, interactivity and usability.

But that doesn't mean that we should throw away the Web's hard-won advantages of interoperability and scalability, that we perhaps take too much for granted.

Enter the U-Web! The U-Web "puts the Web back into Web 2.0 and Mobile 2.0".

It takes the best of the Web's one-way static document publishing model, and extends it to a two-way dynamic data exchange model, while keeping the interoperability and scalability of the Web.

Instead of working on widget standards that break the Web, let's standardise a fully Web-compatible Mobile 2.0 architecture that delivers the same rich, personal functionality, but adds back the seamless mashability of ever-changing people and their ever-changing stuff. Oh, and promises scalability and rapid, easy development.

I've kicked off the project with the U-Web proposal - perhaps you'd like to jump in and help? Email me (see left bar) or leave a comment.

The Mobile 2.0 Killer App is the App Killer

2008-12-19T17:05:00Z

Mobiles are unique - if you want to miss out on the opportunity they represent, you could choose to see them as just slow computers with tiny interfaces and dodgy Internet connections. Then try to squeeze in your traditional applications; try squeezing the office desktop metaphor with its sedentary documents into a device the size of a mouse!

Alternatively, see them as the most personal, social and dynamic of devices that are becoming connected to the Internet. Now a multi-billion-scale global opportunity opens up to you. That's customers and dollars! In trying to grasp this, some are calling it 'Mobile 2.0', by analogy with its sibling, Web 2.0.

In that light, the Killer App for Mobile 2.0 is the sharer, masher and updater of People, Things, Times and Places... The key to getting Mobile 2.0 right is for it to merge seamlessly into our lives. That means the handling of dynamic and shared data becomes the top priority, even above the handling of applications.

This article describes a Mobile 2.0 platform that makes people and their stuff first class - not applications.

Mobile 2.0 should be about effortless delivery of dynamic data such as local weather, personalised news feeds, timelines off Facebook and feeds off Twitter, messages, notes, videos, photos, calendar entries, map locations and meetup plans.

Mobile 2.0 should easily allow us to two-way-share this dynamic data with family, friends, contacts, third parties and the world.

People don't actually want applications on their mobiles!

What Mobile 2.0 does not need is traditional applications, because applications just get in the way of this sharing, mashing and updating of live and personal data. What most people want on their mobiles is not the applications, but the stuff they animate.

People only accept the concept of applications (whether a native app or a Web app) because that's all they've been offered, and it's largely good enough. But no-one actually wants to download and launch and register and log in to a local find-your-friends application - they just want to find their friends in the area - now! And they shouldn't then have to flip between the find-your-friends map owned by that application and the restaurant review map owned by another.

They don't want Facebook videos and YouTube videos and phone videos. They just want to share videos. They shouldn't have to think about whether to send a picture by MMS or to use an upload app, after remembering the login. They don't want multiple ways of sending messages: IM, SMS, Twitter, Facebook, etc. They shouldn't have to think about how to tell their friends about some news item - whether to post a TinyURL link on Twitter or copy the text manually into Facebook.

They only want one shared calendar, not the phone calendar and a Google calendar and events on Upcoming.org, that need two more logins. They shouldn't have to think about how to synchronise music or contacts lists on the phone, the iPod, the PC, some memory card and online.

Applications - both native and Web, and including Applets and Widgets - just get in the way of life! They try to own or control our People, Things, Times and Places, breaking them up with arbitrary application boundaries and making things unnecessarily hard to do.

People just want to share new stuff through their mobiles

What most people would prefer is for their friends and messages from their phonebook, their Facebook, their Twitter, their IM, plus their Flickr photos, their Facebook photos, their restaurant and film reviews and their meeting places all to appear on the same map or in the same calendar... along with the local weather forecast for now and for any dates being planned.

And for that map or calendar, those photos, messages, interesting news, etc., to be actively shared amongst those friends, or even the entire world, simply by flicking the 'share with friends' or the 'make public' switch. Any member of a group should be able to post photos into a shared gallery, or scribble onto a shared whiteboard.

The Killer App for Mobile 2.0 is the sharer, masher and updater of People, Things, Times and Places. This Mobile 2.0 platform would seamlessly merge, update, synchronise, upload, download and save your latest life stuff, including that of your friends and family, so that you no longer have to think about doing it yourself.

Your identity and your stuff would be owned and controlled by you through your personal handset. They would not be merely transferrable - they'd simply be transferred, to where they're needed, when they're needed, using an open Internet protocol and standard formats.

What this Mobile 2.0 platform must do

This platform will have a tactile 3D interface, with touchable, draggable areas for People (friends, family, organisations), Things (photos, videos, messages, feeds, news, notes), Times (appointments, plans, weather) and Places (maps, galleries). It will seamlessly access and present this interactive content from the phone itself - representing you - from friends, and from third parties.

This interactive content will be findable by tags, geo-tags and timestamps - it's semantic data, not document text. You'll be able to pick a Person, then see their related Things, Times and Places; or pick a Thing, to see it's associated People, Times and Places. Stuff will also be findable by recommendation and trust, or at worst by well-targetted advertising.

Loss of Internet will be handled gracefully - stuff from others will still be visible, but simply not updated. There are no central servers, so loss of a server only affects the particular content it sources and 'animates' for you.

Third parties will easily be able to add functionality and interactive content such as map overlays, media, news feeds, etc.. They just expose them using the open protocol and notation - they won't have to ask anyone in order to do so. Some contributors can adapt the Facebook, Flickr, Twitter and various IM APIs in the same way. This will require a system for reconciling the multiple identities of users and their contacts.

There will need to be a virtual property system backed up by crypto keys and supporting payment - just because we don't have an App Store, doesn't mean we can't charge for interactive content! It will also be possible to charge for premium services, such as remote storage space for over-enthusiastic photo-snappers.

Blueprint for the Mobile 2.0 Killer Application

Right, we need to describe some more important things with capital letters, because we're going to talk about the best open notation and Internet protocol for sharing, mashing and updating...

We've already introduced People, Things, Times and Places. Call these Entities. Entities are open data - they have a current State. We'll use a text-based notation to describe their public State, because it's easy to read and write. Each Entity will conform to an open standard public schema or syntax.

Entities can Link up because they will each have a UID (unique identifier). Some friends-People met at the last-week-Time and the Tower-of-London-Place and took these photo-Things. I have a collection Thing of People and both personal and public collection Things of photo-Things. You get the idea. That's a form of mashing. Oh - photos on the Web can be wrapped in Entities that pull out their metadata, and Web URLs are valid Links.

Any Entity can Observe another. Well, it's mostly People Entities doing the Observing and being Observed, via the mobile device: my Person Observes your Person to see what you're up to, then Observes your collection Thing of latest photos and may watch you on the map Place as you head towards me snapping more. Sharing and updating.

But some Things can be Observers, too; the more interactive ones that are affected by Entities around them. A simple example may be a restaurant review summarising Thing, that Observes and averages the scores off several reviewers' submission Things. Each time a score is added or changed, the final score adjusts accordingly. That's sharing and updating, too, and Entities that depend on other Entities is another form of mashing.

Entities have read permissions, which control which People and other Entities can see their State. Crypto tech may be needed to support this in some cases. Entities don't have write permissions as such, because they control their own State as a result of what they Observe.

Of course, these Observations occur over the Internet. This will use an open standard publish-subscribe, peer-to-peer protocol. An Internet packet will subscribe to an Entity's Link, another packet will return current Entity State, subsequent packets will broadcast State updates to current subscribers. State is transferred into servers and handsets as required, by a combination of pull and push - sharing and updating. The protocol will ultimately also have to support things like streaming and real-time media; over Wifi at least, so as not to scare off the carriers...

The U-Web Mobile 2.0 platform

This Mobile 2.0 Killer Application for sharing, mashing and updating People, Things, Times and Places ... will have a shorter name! I'm calling it the 'U-Web' Mobile 2.0 platform at the moment, reflecting its affinity to the Universe Web I described before. The U-Web platform application will be free and the protocols and formats outlined above will be open.

The typical handset specification and APIs we're aiming for are: at least 320x320 touch screen with Embedded 3D; GPS, Wifi and good 3G; front- and back-facing cameras.

There's a large number of Windows Mobile devices that match this, including phones from HTC, Sony-Ericsson, Samsung, etc. There's currently one Symbian (the Nokia 5800/Tube with the N97 to come next year). One iPhone. One Android. One Blackberry without Wifi (the Storm).

Although Nokia's plans are somewhat unclear, their S60v5/Qt/Ovi offering looks like it may preclude the U-Web, especially since the U-Web wants to be the top app... The U-Web certainly wouldn't be allowed onto the Apple App Store - the clue is in the name! Writing it in C also rules out Java phones, such as Android - which in any case is another app-oriented platform - and any Blackberry handsets.

The U-Web should thus be written for Windows Mobile 6+ Professional handsets. At least their users will have a ready appetite for anything integrated and easy to use, that keeps them clear of the .. clumsy .. operating system. It will also be worth keeping an eye on the various Linux MID (Mobile Internet Device) platforms, including OpenMoko, Maemo and Moblin, and maybe LiMo. Similarly, we should write for Windows and Linux on the server side - the code will be essentially the same. It'll need to be written in C as the lowest common denominator, most efficient language.

The U-Web will be the top or idle application: on Windows Mobile it would replace the Today screen, or more likely TouchFLO 3D or Spb Mobile Shell. Else it could be an Xperia X1 Panel. Like these, it will give access to phone functions such as calls, texts, camera, speaker mute, Bluetooth and Wifi enable, lock screen, brightness, etc.

Of course, it won't actually be the only application while it is in its beta phase - people may still want to access a browser, email and various settings and utilities. The need for browser and email should reduce significantly when functions such as Web 2.0, messaging and file sharing are drawn up into the mobile-native U-Web, leaving just actual documents fitting awkwardly.

Opportunity

It is a mistake to see Mobile as just a second-class computing platform or as a miniature office desktop onto which you must squeeze applications and documents.

Mobile represents a whole different world of personal interaction and almost 100% presence - especially now that mobiles are becoming Internet-connected and location-aware and are being furnished with top-quality cameras and touch screens. The mobile device represents you, here, now; what you're seeing and doing, to your social network.

A Mobile 2.0 platform needs to drop closed applications and go instead with the flow of People, Things, Times and Places. It needs to transparently share, mash and update people and their stuff.

The U-Web Mobile 2.0 platform described here is designed to seamlessly merge your real and virtual lives, starting with intuitive, 3D touch interfaces that give an almost tangible feel to virtual property, then expanding out to touch the 3D lives of others.

If you take a photo, your friends can instantly 'join you on the map' where you stand and chat about it - and the weather conditions you're experiencing. No need to upload to photo sharing sites with logins, or to launch isolated chat, map and weather apps.

The opportunity is billion-scale - increasingly capable mobile devices will become the world's primary connected computing device, outstripping PCs and making current Internet and Web usage look like just a warm-up.

We should be in there with appropriate software when that happens. We can stumble into this step-by-painful-step via applications and browsers, or we can just let go and meet the actual requirements - with a platform for mashing dynamic, shared data.

I'd be delighted to hear from you if you're interested in helping or getting involved in any way with building this platform! Email me - link's on the left - or leave a comment.

Google Micro Conference

2007-10-05T11:22:00Z

Last night's Google London Open Source Jam (also here) was on the subject of the 'Web' (didn't they invent that? Oh no, that was Microsoft).

This event has been getting better and better each time I've attended. There were some very interesting lightning talks held together with a tight structure and plenty of chance to chat, drink cold Leffe and eat cold pizza. And nick [transatlantic translation: 'steal'] the Green & Black's chocolate.

An ideal Micro Conference...

I arrived late (it starts at 6pm) and spent some time catching up with ex-Thoughtworks colleagues, so I missed Dion "Ajaxian" Almaer's Google Gears slideset from FOWA. Go there now and check it out.

Thus the first talk I saw was a nifty piece of widgetry by Steven Goodwin called WARP. In WARP, interacting with a page of 'applets' changed the URL to encode those applets' current state. If you link to the current page, it will always show that state. Very long URLs, you can imagine. None of that fancy Ajax stuff. RESTful, dare I say. Nice API server-side for unpacking your applet params.

A trip to the lavatories [transatlantic translation: 'restroom'/'bathroom'] revealed that they are, indeed, doing that Testing in the Toilet project in Google. It works, too! I learned something. Other intelligence on Google's Inner Workings include confirmation of the beanbags and of the high quality, free grub to which I have already alluded.

A nice bloke from Yahoo! (Tom Hughes-Croucher: another spy?) came along to sell his idea that, in the collaborative world of open-minded hackers, we who run websites could help each other with our 404s. If I get a 404, I use the referrer link to tell you, via some RESTful POST, that your link to me is bust (assuming I don't intend to fix it myself).

I think the world is a little more selfish, so you need to decide who hurts more - the site who sends their visitors to a dead-end, or the site delivering that dead-end to a new visitor. I suspect the latter, by a small margin, as it's not exactly a nice welcome. So it's up to them to let the new visitor down more gently, and to notify the publisher of the broken link with little or no cost to them. For example, a really sociable 404-ing site could just redirect the hapless visitor back to the referring page, adding '?broken=links' to the URL - hopefully to be picked up by log scanning scripts at the referring site.

Next up, yours truly taking yet another chance to promote his excellent Micro Web thingy. Couple of people asked about it afterwards - including that nice chap from Yahoo! Also, a smart - and nice - chap called Toby (this one?) got me into a deep discussion on imperative vs. event-driven vs. state-driven programming. He was apparently an old-timer like me, as he was able to engage in dewy-eyed Functional Programming recollections. I managed to give out about four full colour printouts about the Micro Web, and to collect some good calling cards.

However, Joe Walnes, even a pint down in the pub afterwards, still refused to sign up for Micro Web duties. This in spite of over three years of intensive lobbying, including eight months of me working Trojan-horse-like in his kitchen, on The 2005 Implementation.

Another ex-Thoughtworks colleague, Simon Stewart took yet another chance to promote his promising Webdriver thingy. And a very interesting project it is becoming. Still needs more work - on IE support, etc - but I'll probably be using it in my new job at the Financial Times.

Another ex-Thoughtworks colleague, Chris Matts took a chance to promote his and Andy Pols' interesting new Dream Machine thingy. Perhaps a bit like Cambrian House - you put your dreams and ideas into it and people expand on them. Chris is a natural on-stage - and even used the age-old trick of promising lots of money for no effort, to get our attention at the start.

All I could come up with for the Micro Web was 'Cheaper, Wider, Faster'...

Updated: added reference to Dion Almaer, details about WARP, swapped in the picture of TWers that I was waiting for and fixed a minor blunder thanks to that ever-sharp ThoughtWorker, Dan Bodart..

How Ruby can enable the Web 2.0 Platform

2007-06-26T15:17:00Z

Web 2.0's definition includes seeing the Web as an application platform. Which means it is in competition with Java and .Net, and with SOA, for both local and widely distributed applications.

If the Web is going to be a platform, the skills you need to learn to program it are the core Web 2.0 technologies such as Ajax, JSON, Atom, Microformats and OpenID.

And Ruby. This language, that's capturing the hearts of many Web 2.0 programmers, is ideal for easing the transition from the Java and .Net platforms to the Web platform, as I will show.

Even if you're part of a big company that is generally immune to the latest trends, the marriage of Ruby and the Web-as-platform may be something to prepare for. It could even displace your SOA agenda...

Few would disagree that the Ruby language is riding the wave generated by Ruby-on-Rails. In turn, Rails is riding the Web 2.0 wave, coming as it does from underpinning the very Web 2.0 37signals product suite.

Rails and Ruby have tapped into the tech Zeitgeist of friendly, simple and powerful. The speed with which the Ruby and Rails communities have delivered the key components of Web 2.0 is matched by the speed at which Ruby and Rails books are leaving the shelves.

What is the ideal platform of Web 2.0? Will it be Rails and Ruby? Will Ruby ride the Web 2.0 wave into the mainstream in the same way Java rode the Web 1.0 wave?

Well, here's the problem with that question: Web 2.0 is supposed to be primarily about the Web itself as the platform, as explained first by Tim O'Reilly and then by a thousand Web 2.0 vendors and industry watchers after him.

Web-as-platform is not just vendor hype or pundit hand-waving. Let's think about what O'Reilly meant by that.

Web as Platform

Web 2.0 is about making the Web more interactive, and thus able to support applications where Java and .Net would once have been considered the sole delivery platforms.

The fact that the technologies of the Web can be turned to this use is a shift with far-reaching implications.

Broadly, the shift we are seeing is from the one-way, static document delivery of Web 1.0 towards the two-way, dynamic data exchange of Web 2.0.

This fundamental repurposing is delivering more complex, interactive applications that work inside our browsers and which fully leverage the benefits of online operation.

Web 2.0 is bringing the user and their stuff into the very Web that they hitherto only passively consumed. This network-enablement of the user in turn enables their social networking and their shared creativity and self-expression.

Web 2.0 has tapped into a deep human need - a fact reflected in the vast traffic volumes and correspondingly vast valuations of Web 2.0 startups that we're currently seeing.

But Web 2.0 is not just for the startups: Enterprise Web 2.0 is coming! The bigco.com site is going to be looking a little, well, static and lifeless when compared to the new sites that are springing up everywhere, and that most of BigCo's employees are using. Further, BigCo can gain huge benefits from Web 2.0 approaches empowering and connecting those employees on the Intranet. And that Intranet is an ideal platform for deploying company-wide, interactive applications.

This shift in the Web to two-way dynamic data is being powered by a set of technologies that a Web platform programmer is going to have to learn.

Web 2.0 Platform Technologies

Anything that claims to be an application platform must support data. Web 2.0 is above all the data Web. Web 2.0 is about semantics, not free text and font sizes. Hence, it inevitably starts with data-oriented formats such as XHTML, YAML and JSON. In Web 2.0 more than ever, we talk about data not documents and about separating data from its presentation. CSS is big in Web 2.0, for good reason (not just for gradient fills). Inside the page of a self-respecting Web 2.0 application, you'll often find Microformats - again, semantics in the page: publishing concise data of widely-understood standard formats. Some of those Microformats may be tags, and in Web 2.0 the simplest and most powerful semantics are those little pivot points in Webspace.

Again, if you're going to be a general purpose platform, you need to be able to fetch, update, notify and display that data. Web 2.0 integration usually happens via JSON data structures and REST interfaces (some of which, especially those based on AtomPub, are true REST). Following on from the data-like pages we serve to browsers, come the data-like feeds we publish to feed readers and to other applications. After feeds, the core technology that gives Web 2.0 its dynamism and interactivity is Ajax and DHTML, and increasingly Comet (server push to the browser). The core technology that gives Web 2.0 its users is increasingly OpenID.

All of the above are open technologies. You can do Web 2.0 without proprietary technologies, just like Web 1.0. Indeed, keeping to the principles that made the Web successful is also essential to the success of Web 2.0. The Web platform is the first application platform that has to consider scalability and interoperability, and will ignore them at its cost. I have written before about open data, use of standard data formats and using REST properly to avoid creating unscalable, walled-garden sites. You don't need Flash or SilverLight, you don't need vast amounts of custom Javascript, you don't need function calls tying you to your servers.

Programming the Web 2.0 Platform

So, we've got the dynamic data that you'd expect of a would-be platform. But how to drive changes in those data? How do we program the Web platform to animate all this data?

All Rails programmers will know the above technology list; it comes with the territory. The Web 2.0 Platform can be very succesfully powered by Rails and Ruby. Ruby and Rails make Web 2.0 applications simple and quick to program, addressing many of the needs of simple Web 2.0 applications out of the box. There's little doubt that Ruby and Rails will have a secure future riding the Web 2.0 wave.

However, for many Web 2.0 applications, programming may not even be necessary, at least not in the procedural or imperative style programmers expect.

Look back to the early 90's: 'Web 1.0' made a whole class of applications easy to write without programming: applications for navigating information. You just wrote in HTML, declaratively.

Now look back at the long path of evolution of Java, through J2EE, Spring, AOP, IoC, Domain Driven Design, POJOs. All trying to achieve the simple goal of 'remove all that MVC and persistence stuff and let us concentrate on business or domain objects'. But they never quite seemed to get it right.

But then Rails comes along, and has succeeded by simple virtue of concentrating on easy manipulation of the 'Intel Inside' of Web 2.0 - data.

It's reminiscent of the 'Naked Objects' approach to application building with minimal programming (just business or domain code in POJOs that expose state into the GUI and are transparently persisted). The Streamlined project takes Rails even further down this path. Rails' nearest competitor, Django, has an admin interface that works in a similar way, automatically generating edit pages based on the data model.

Web 2.0 is about data, about semantics. Web 2.0 is inherently declarative. So Web 2.0 applications can be written declaratively - Web 2.0 mashups can be just wired together and their data animated by business rules. A bit like programming spreadsheets.

Teqlo, Coghead, Pipes, DabbleDB, LongJump, Popfly, AppExchange and Wyaworks are all examples of the different ways to program the new Web 2.0 platform without imperative code.

That's what we mean by Web-as-platform - not only is the underlying programming language irrelevant, it will often not even be needed, certainly for simple data manipulation applications and for many simple mashups. Being RESTful gives you a massive head start in this, of course.

While Rails is already in the game with its innate understanding of Web 2.0 techniques and philosophies, Ruby itself has a huge amount to offer the would-be declarative programmer, who is making the transition to this new Web platform from their traditional Java or .Net platform. In particular, it is easy to write your domain logic in a declarative style in Ruby: they call them 'DSLs' these days, but the idea is the same in most examples I've seen.

Web 2.0 - The Web Redux

Now, if you've been following this blog, you'll know I have a few opinions on Declarative Web 2.0 and on patterns for programming REST. Essentially I argue that, if you want to play in the Web 2.0 platform game, you don't want to be writing screeds of Javascript functions that call more functions on your servers.

I recently presented some ideas along these lines at the WWW2007 conference, entitled 'The Micro Web: putting the Web back into Web 2.0', where I also showed a demo written in Python.

This approach combines my Distributed Observer Pattern with Comet push to enable highly dynamic Web 2.0 applications to be coded RESTfully and declaratively, with zero Javascript. The Distributed Observer Pattern offers a clean programming model for animating the Web 2.0 dynamic-data technology set I described above.

I believe the Observer Pattern is core to the way we'll be programming when the Web 2.0 Platform hits mainstream. It enables the kind of event- and rule-driven programming that matches the characteristics of the Web 2.0 dynamic data platform. As a further killer benefit, it also directly addresses the optimal utilisation of multicore processors.

I am currently porting my Python implementation of this approach to Ruby, in the Redux project on Rubyforge. Redux stands for 'Ruby Event-Driven Update Exchange'. It uses the highly scalable EventMachine epoll-based event loop to power its event-driven architecture. This will be essential when Redux is asked to scale up a Comet-based application.

Like Rails, Redux will be a Web (2.0) application framework, but unlike Rails, it puts the Observer Pattern and event- and rule-driven programming at its core.

Redux's headline is 'Web 2.0 in-a-box' or 'Naked Objects on the Web'.

Conclusion

If you're in BigCo, and are responsible for setting BigCo's technical strategy, then train your Java devs up on Web 2.0 core technologies such as Ajax, JSON, Atom, Microformats and OpenID.

And fire up their enthusiasm by tapping into Ruby (perhaps via JRuby) on your way to the Web 2.0 platform.

Learn patterns for mashing and integrating. Learn about REST and event- and rule-driven programming, including declarative DSLs.

When this Web platform hits BigCo, you will probably find that its REST or ROA style make your SOA integration strategy look rather complex and unweildy.

Check out the Distributed Observer Pattern, and download Redux when it's done (I'll let you know if you subscribe here!).

In 2007 and beyond, its the Web itself that's the platform, not Java or .Net. But if you want to get there via a language-based platform, Ruby could be the best way to transition to it.

Note: Everything I said about Ruby and Rails applies equally in technical terms to Python and Django, but regardless of the significant benefits of the latter, Ruby and Rails have the Web 2.0 market and mindshare. I'll probably switch this blog from Django to Redux sometime this year..

Lighter-than Wins in 2007

2007-01-18T11:12:00Z

What do all the MAJOR Web 2.0 technologies of 2007 have in common?

Let me list them first:

    M.icroformats (including tags)
    A.jax (including Comet)
    J.SON (plus YAML)
    O.penID (plus SXIP, LID, Yadis)
    R.EST (including Atom, APP)

What these technologies have in common is that they're all lighter than their competitors:

Microformats	Lighter than the Semantic Web
Ajax	Lighter than Fat Client (!)
JSON	Lighter than XML
OpenID	Lighter than SAML/Liberty Alliance
REST	Lighter than SOA

I'm reminded of Tim Bray's Technology Predictor Success Matrix and his appreciation of the 80:20 phenomenon. If hitting the sweet spot of achieving a lot with only a little is the main test of potential, then these five Web 2.0 technologies are all set for success.

So it's no surprise to see Tim supporting Microformats once or twice and even JSON, in spite of his, um, XML background. He also understands and uses Ajax and backs REST (in particular via Atom). Tim has not yet mentioned OpenID, instead referring to SAML and Liberty Alliance (please don't be crass by leaving a comment speculating on the reason for this...).

These MAJOR Web 2.0 technologies should form the basis of the two-way, interactive data Web or the Web-as-a-platform vision that we've all been promised.

However, they will need taking a little further this year, in order to bring them closer together and to make this vision happen in a seamless way.

A synergy between these 80:20 technologies can potentially produce much more than just the 'next version of the Web', if engineered well.

I have already discussed the Subversive Microformat, I've promoted Declarative Ajax and currently I'm slowly working up to Symmetric REST.

Next up, I'll be asking you to consider a web of JSON Microformats and ways to merge hCard and OpenID into active user personas - even avatars...

I'll tag this acronym as 'rajmo' instead of 'major', for obvious reasons.

Setting Data | The REST Dialogues

2006-11-15T23:37:00Z

In an exclusive nine-part dialogue with an imaginary eBay Architect, we present an accessible discussion of the REST vs. SOA issue.

Although eBay have what they call a 'REST' interface, it is, in fact, a STREST interface, and only works for one of the many function calls that they make available via SOAP (GetSearchResults).

In this dialogue series, I argue the case for eBay to adopt a truly REST approach to their integration API.

Part 2: Setting Data

Duncan Cragg: Now, let's look at those calls in eBay's SOAP API that are expected to change data. There is often a corresponding 'Get' function.

Examples are GetTaxTable, SetTaxTable and GetMyMessages, ReviseMyMessages, ReviseMyMessagesFolders, DeleteMyMessages, etc.

Here, there is an implicit model of data (or lists of data) that you can look at, modify, delete, etc.

eBay Architect: It's not just data: Tax Tables and Messages have meaning and behaviour internally to the server.

DC: Yes, but we're talking about what the API is telling us, here. It says things like GetTaxTable and SetTaxTable!

We'll come on to business processes later, but just look at the simple stuff first.

eA: OK, so what's the benefit of doing these calls in HTTP, through POST or whatever?

DC: Again, scalability and interoperability.

In the same way as with GET, you can partition your POST handling across the URI space. Have your Tax Tables and Message folders handled by many machines, split by their URIs. This is an example of the inherent parallelisability of the Web's architecture.

eA: We do partition by function: search, display, sell.

DC: Sure, but you had to 'hard-code' this partition: to use specific heuristics and code to achieve it: URI-based partitioning is much more generic and flexible, and can be much finer-grained. You seldom get single-point of failure or bottleneck issues with good REST architectures.

eA: If you say so.

DC: Interoperability again derives from the standardisation of the Content-Types and the schemas of POSTed data. When you know the meaning of the data you fetch, you also know how to try and change it, if you can.

eA: In HTML, you actually get presented a form in some GET data which explicitly tells you what to POST back - what the 'schema' is.

What's the equivalent in a general REST integration context where it's machines that are doing the POSTing?

DC: Well, the same applies, insofar as the GET data effectively tells you what any return data should look like. However, this time it's in an implicit way.

Where, in the read context, you understood what to do with a given content type you'd fetched, now, in the write context, you further understand what you can send back to it again.

eA: Can you give examples, please? The Tax Table perhaps?

DC: Straightforward data updating is the simplest case - it's the same content type coming out as going back in. You see a Tax Table at some URI, then you just PUT a new Tax Table back at that URI to try and replace it.

All the API function calls beginning 'Set' should probably be implemented this way.

eA: OK, so how does REST implement our Message API functions?

DC: An example of a data format that implies its edit capabilities by reference to a standard is Atom syndication.

You can GET a list of Atom entries, then POST a new one according to the Atom Publishing Protocol (APP) specification. You POST, not to a 'service' or handler, but to the URI of the actual collection you're adding to.

eA: So what's that got to do with our Messages?

DC: The eBay Message lists are a great candidate for using this approach, with the benefit that any (recent) feed reader can be used to view them, and an APP-compliant client used to manage them.

All the API function calls beginning 'Add' should probably be implemented in the same way APP adds new entries, or you should consider using APP itself to implement them.

eA: OK, so we can replace some calls with a standard set of REST interactions. But clients still have to know about Tax Table schemas, User Preferences schemas, Message schemas, etc.

It's not enough to simply know about HTTP and HTML to be interoperable any more, like a browser does. It's not even enough to just know APP.

Surely interoperability stops when the content types and schemas start to proliferate like this?

DC: Schema proliferation is SOA's problem - so I'm glad you brought it up! It's baked in to the SOA mindset that it's OK to design your own interfaces and schemas from scratch each time.

Conversely, the expectation and culture in the Web and especially in the REST-aware community is to constantly look out for opportunities to conform and to standardise schemas and interactions, and to build on layers below that are already standardised. It's a side-effect of the shareability of URIs.

With REST you get interoperability at many levels above the byte-transfer of HTTP. Content type understanding and standardisation can occur all the way up from characters to Tax Tables.

eA: I'm not sure I get this.

DC: OK: there is some code that understands basic HTTP GET and PUT, and UTF-8 characters - and doesn't need to know what a Tax Table is - some that understands UTF-8 plus XML, some that understands that plus Atom, or plus XHTML, then XHTML tables and Microformats like hCalendar, hCard (and hAtom!), then conference schedules using hCalendar and hCard. Tax Tables can perhaps use XHTML tables.

Clients may or may not need to know what the schema of a Message looks like internally in order to be able to do useful work with Messages at their level of understanding - from character stream up to APP.

eA: I'm not sure how general auction schemas fit in there.

DC: You never know, your eBay schemas might be taken into account when coming up with new standard content types for e-commerce!

eA: Hopefully. We could always put everything into XHTML tables for you!

DC: Ha! Go for it. Having done some HTML scraping myself, I'd welcome such semantics!

eA: Right, it's OK when we're just talking data that you can read and write - HTTP, APP, REST, whatever may be great for that - but our API has much more complex business functions in it that don't fit that simplistic model.

There's a whole load of functions in our API that aren't just about getting and setting data or collections of data. Some are eBay- or auction-specific.

DC: This is one of the Myths of REST - that it's just for simplistic reading and writing of data.

It's a actually a myth that's often propagated by the REST community itself, especially with their over-emphasis on the Four HTTP Verbs, which seem to map so conveniently onto Create, Read, Update and Delete.

eA: But they do map so well - I thought that was 'good' REST...

DC: It is often good, but can detract from the whole goodness!

One consequence of this mapping it that people inevitably go on to see an analogy between REST and databases, and then start to expect transactions and other database features.

Another consequence of the database analogy is that resources are seen as lifeless servants of the active client: it takes away responsibility from a resource to be master of its own destiny.

This then causes confusion (even within the REST community) about the power of the client and even the very meaning of such basics as the PUT method.

The fact is, GET and POST are more than enough to be REST compliant. This cut-down pair also help focus the mind on URIs, two-way content transfer, content type or schema and on the responsibilities of each resource as active players in an integration scenario.

eA: Sounds like a minority REST view - any support from a REST luminary for that?

DC: Tim Bray has a history of suggesting that GET and POST are enough.

And recently, there has been further high-level support from Sam Ruby and Leonard Richardson in their manifesto for an upcoming book.

eA: I bet you still get told off, but it sounds reasonable to me.

DC: So, to summarise: use GET to read data, then POST back to the same URI to suggest changes to the resource there.

All based on a given level of understanding and interpretation of the content type and any corresponding interaction standards.

Interoperability and scalability in a nutshell!

Now I will explain how there's more to REST than simple reading and (attempted) writing of resources. We're still only two-ninths done...

In Part 3: Business Functions.

Note that the opinions of our imaginary eBay Architect don't necessarily represent or reflect in any way the official opinions of eBay or the opinions of anyone at eBay.

Indeed, I can't guarantee that the opinions of our real blogger necessarily represent or reflect in any way the official opinions of Roy Fielding...

Getting Data | The REST Dialogues

2006-11-14T00:05:00Z

In an exclusive nine-part dialogue with an imaginary eBay Architect, we present an accessible discussion of the REST vs. SOA issue.

Although eBay have what they call a 'REST' interface, it is, in fact, a STREST interface, and only works for one of the many function calls that they make available via SOAP (GetSearchResults).

In this dialogue series, I argue the case for eBay to adopt a truly REST approach to their integration API.

Part 1: Getting Data

Duncan Cragg: So - let's get straight to my argument: I claim that your SOAP APIs, as instances of the SOA style, won't scale or interoperate as well as they would if they were implemented in the REST style. Which, in the form of the Web, has largely proven scalability and interoperability.

eBay Architect: SOA won't scale or interoperate and REST will? You've got some explaining to do there: the whole industry is going SOA crazy, and we're following that trend. You'll need to give specific examples.

DC: OK.

eA: And even if it is true, we don't care about scalability and interoperability: we have quite low traffic on our APIs - certainly compared to port 80; plus we're eBay so we don't need to interoperate with anyone.

DC: That's true now, but if the Web 2.0 vision comes together, you may care: your API traffic could increase dramatically. It would be better to be the one prepared for the scale of the API-Web!

Can you really argue in your company that you don't need to be scalable? What if your port 80 traffic needs to be routed to your APIs for some reason?

eA: Maybe.

DC: As for interoperability, you could be excluded from Web 2.0 industry-boosting consortia, or excluded from perhaps hugely popular Web 2.0 applications in the future...

Interoperability raises the level of the market as a whole. Market players shouldn't differentiate on what's common to them, they should differentiate on the level above.

It also depends on the value you place on having happy customers who don't have to do the same thing multiple ways or multiple times.

eA: Ha! I'm sure there's a name for that style of argument!

So, let's say we want to be good scalable and interoperable citizens in a future Web 2.0 world: show me how REST can help us more than SOA and SOAP do!

Getting Data

DC: OK, let's look at your SOAP API. There are 72 function calls in there that begin with 'Get'. Each one specifies a particular piece of data that you can fetch.

eA: Yes, our API is extremely rich.

DC: Sure, but you don't need a new function call for everything you can get from your system: you can just use HTTP GET!

eA: OK, but it's just data going in to request the information, then data coming back out again. What difference does it make whether we use HTTP or SOAP to carry those messages? You've just moved the name onto your URI!

DC: It's not just any 'data going in': the URI can be passed around for anyone to re-use. This URI is more interoperable because so much deployed software understands it. No-one understands 'GetSearchResults()'!

eA: OK.

DC: Another example of how the URI can glue things together is that the data returned from your GETs can have more URIs in them, ready to go! You won't get data from your Web Service with 'GetItem()' in it..

eA: Heh - I s'pose not.

DC: REST also talks about the formats of the data behind a URI. In a GET, the response data is given a Content-Type, and there's an expectation that clients will understand the types of data being returned: interoperability comes from broad standardisation of return data.

eA: But we return XML, and our schemas are published. They're specific to our application anyway. That wouldn't change with REST.

DC:The explicit statement of Content-Type reflects a culture of agreement forced by the sharability of URIs: your URIs are more sharable when more clients understand the data they dereference to.

On the other hand, the culture of SOA is to declare custom WSDL and custom XML schemas.

Like I said, one day you may care about interoperability, and having an architecture that puts a high value on content type and schema standardisation, as REST does, puts you one step ahead.

eA: We'll see.

DC: You can also gain scalability by partitioning on those URIs.

eA: We already do plenty of partitioning of our data and functions.

DC: Yes, but URI partitioning cuts right through the system in a very simple way: your partitioning is an application-specific optimisation which has to be hand-coded behind the SOAP interface.

eA: Mmm. Maybe.

DC: Another benefit of using HTTP over using SOAP is that you get cacheing built in to the architecture, which you can start using as soon as you ask for it in the headers. This boosts scalability.

eA: OK, that's fair. Our clients would have to write their own caches, should they need them. But in general we advise against cacheing our data and don't put expiry on them or have not-modified checks.

DC: Which is where you're potentially inefficient.

eA: Maybe. We internally cache things like the data behind GetCategories, and advise our users how to cache it themselves.

DC: Again - it's application-specific.

So - even in the simple cases of fetching data, REST has given you much greater scalability and interoperability than your SOAP interface - as well as a simpler, more generic approach.

And we're only one-ninth of the way through our conversation!

eA: How do you know that??

DC: Aaah...

In Part 2: Setting Data.

Note that the opinions of our imaginary eBay Architect don't necessarily represent or reflect in any way the official opinions of eBay or the opinions of anyone at eBay.

Indeed, I can't guarantee that the opinions of our real blogger necessarily represent or reflect in any way the official opinions of Roy Fielding...