Web 2.0's definition includes seeing the Web as an application platform. Which means it is in competition with Java and .Net, and with SOA, for both local and widely distributed applications.
If the Web is going to be a platform, the skills you need to learn to program it are the core Web 2.0 technologies such as Ajax, JSON, Atom, Microformats and OpenID.
And Ruby. This language, that's capturing the hearts of many Web 2.0 programmers, is ideal for easing the transition from the Java and .Net platforms to the Web platform, as I will show.
Even if you're part of a big company that is generally immune to the latest trends, the marriage of Ruby and the Web-as-platform may be something to prepare for. It could even displace your SOA agenda... ...
Web 2.0's definition includes seeing the Web as an application platform. Which means it is in competition with Java and .Net, and with SOA, for both local and widely distributed applications.
If the Web is going to be a platform, the skills you need to learn to program it are the core Web 2.0 technologies such as Ajax, JSON, Atom, Microformats and OpenID.
And Ruby. This language, that's capturing the hearts of many Web 2.0 programmers, is ideal for easing the transition from the Java and .Net platforms to the Web platform, as I will show.
Even if you're part of a big company that is generally immune to the latest trends, the marriage of Ruby and the Web-as-platform may be something to prepare for. It could even displace your SOA agenda...
Few would disagree that the Ruby language is riding the wave generated by Ruby-on-Rails. In turn, Rails is riding the Web 2.0 wave, coming as it does from underpinning the very Web 2.0 37signals product suite.
Rails and Ruby have tapped into the tech Zeitgeist of friendly, simple and powerful. The speed with which the Ruby and Rails communities have delivered the key components of Web 2.0 is matched by the speed at which Ruby and Rails books are leaving the shelves.
What is the ideal platform of Web 2.0? Will it be Rails and Ruby? Will Ruby ride the Web 2.0 wave into the mainstream in the same way Java rode the Web 1.0 wave?
Well, here's the problem with that question: Web 2.0 is supposed to be primarily about the Web itself as the platform, as explained first by Tim O'Reilly and then by a thousand Web 2.0 vendors and industry watchers after him.
Web-as-platform is not just vendor hype or pundit hand-waving. Let's think about what O'Reilly meant by that.
Web as Platform
Web 2.0 is about making the Web more interactive, and thus able to support applications where Java and .Net would once have been considered the sole delivery platforms.
The fact that the technologies of the Web can be turned to this use is a shift with far-reaching implications.
Broadly, the shift we are seeing is from the one-way, static document delivery of Web 1.0 towards the two-way, dynamic data exchange of Web 2.0.
This fundamental repurposing is delivering more complex, interactive applications that work inside our browsers and which fully leverage the benefits of online operation.
Web 2.0 is bringing the user and their stuff into the very Web that they hitherto only passively consumed. This network-enablement of the user in turn enables their social networking and their shared creativity and self-expression.
Web 2.0 has tapped into a deep human need - a fact reflected in the vast traffic volumes and correspondingly vast valuations of Web 2.0 startups that we're currently seeing.
But Web 2.0 is not just for the startups: Enterprise Web 2.0 is coming! The bigco.com site is going to be looking a little, well, static and lifeless when compared to the new sites that are springing up everywhere, and that most of BigCo's employees are using. Further, BigCo can gain huge benefits from Web 2.0 approaches empowering and connecting those employees on the Intranet. And that Intranet is an ideal platform for deploying company-wide, interactive applications.
This shift in the Web to two-way dynamic data is being powered by a set of technologies that a Web platform programmer is going to have to learn.
Web 2.0 Platform Technologies
Anything that claims to be an application platform must support data. Web 2.0 is above all the data Web. Web 2.0 is about semantics, not free text and font sizes. Hence, it inevitably starts with data-oriented formats such as XHTML, YAML and JSON. In Web 2.0 more than ever, we talk about data not documents and about separating data from its presentation. CSS is big in Web 2.0, for good reason (not just for gradient fills). Inside the page of a self-respecting Web 2.0 application, you'll often find Microformats - again, semantics in the page: publishing concise data of widely-understood standard formats. Some of those Microformats may be tags, and in Web 2.0 the simplest and most powerful semantics are those little pivot points in Webspace.
Again, if you're going to be a general purpose platform, you need to be able to fetch, update, notify and display that data. Web 2.0 integration usually happens via JSON data structures and REST interfaces (some of which, especially those based on AtomPub, are true REST). Following on from the data-like pages we serve to browsers, come the data-like feeds we publish to feed readers and to other applications. After feeds, the core technology that gives Web 2.0 its dynamism and interactivity is Ajax and DHTML, and increasingly Comet (server push to the browser). The core technology that gives Web 2.0 its users is increasingly OpenID.
All of the above are open technologies. You can do Web 2.0 without proprietary technologies, just like Web 1.0. Indeed, keeping to the principles that made the Web successful is also essential to the success of Web 2.0. The Web platform is the first application platform that has to consider scalability and interoperability, and will ignore them at its cost. I have written before about open data, use of standard data formats and using REST properly to avoid creating unscalable, walled-garden sites. You don't need Flash or SilverLight, you don't need vast amounts of custom Javascript, you don't need function calls tying you to your servers.
Programming the Web 2.0 Platform
So, we've got the dynamic data that you'd expect of a would-be platform. But how to drive changes in those data? How do we program the Web platform to animate all this data?
All Rails programmers will know the above technology list; it comes with the territory. The Web 2.0 Platform can be very succesfully powered by Rails and Ruby. Ruby and Rails make Web 2.0 applications simple and quick to program, addressing many of the needs of simple Web 2.0 applications out of the box. There's little doubt that Ruby and Rails will have a secure future riding the Web 2.0 wave.
However, for many Web 2.0 applications, programming may not even be necessary, at least not in the procedural or imperative style programmers expect.
Look back to the early 90's: 'Web 1.0' made a whole class of applications easy to write without programming: applications for navigating information. You just wrote in HTML, declaratively.
Now look back at the long path of evolution of Java, through J2EE, Spring, AOP, IoC, Domain Driven Design, POJOs. All trying to achieve the simple goal of 'remove all that MVC and persistence stuff and let us concentrate on business or domain objects'. But they never quite seemed to get it right.
But then Rails comes along, and has succeeded by simple virtue of concentrating on easy manipulation of the 'Intel Inside' of Web 2.0 - data.
It's reminiscent of the 'Naked Objects' approach to application building with minimal programming (just business or domain code in POJOs that expose state into the GUI and are transparently persisted). The Streamlined project takes Rails even further down this path. Rails' nearest competitor, Django, has an admin interface that works in a similar way, automatically generating edit pages based on the data model.
Web 2.0 is about data, about semantics. Web 2.0 is inherently declarative. So Web 2.0 applications can be written declaratively - Web 2.0 mashups can be just wired together and their data animated by business rules. A bit like programming spreadsheets.
Teqlo, Coghead, Pipes, DabbleDB, LongJump, Popfly, AppExchange and Wyaworks are all examples of the different ways to program the new Web 2.0 platform without imperative code.
That's what we mean by Web-as-platform - not only is the underlying programming language irrelevant, it will often not even be needed, certainly for simple data manipulation applications and for many simple mashups. Being RESTful gives you a massive head start in this, of course.
While Rails is already in the game with its innate understanding of Web 2.0 techniques and philosophies, Ruby itself has a huge amount to offer the would-be declarative programmer, who is making the transition to this new Web platform from their traditional Java or .Net platform. In particular, it is easy to write your domain logic in a declarative style in Ruby: they call them 'DSLs' these days, but the idea is the same in most examples I've seen.
Web 2.0 - The Web Redux
Now, if you've been following this blog, you'll know I have a few opinions on Declarative Web 2.0 and on patterns for programming REST. Essentially I argue that, if you want to play in the Web 2.0 platform game, you don't want to be writing screeds of Javascript functions that call more functions on your servers.
I recently presented some ideas along these lines at the WWW2007 conference, entitled 'The Micro Web: putting the Web back into Web 2.0', where I also showed a demo written in Python.
This approach combines my Distributed Observer Pattern with Comet push to enable highly dynamic Web 2.0 applications to be coded RESTfully and declaratively, with zero Javascript. The Distributed Observer Pattern offers a clean programming model for animating the Web 2.0 dynamic-data technology set I described above.
I believe the Observer Pattern is core to the way we'll be programming when the Web 2.0 Platform hits mainstream. It enables the kind of event- and rule-driven programming that matches the characteristics of the Web 2.0 dynamic data platform. As a further killer benefit, it also directly addresses the optimal utilisation of multicore processors.
I am currently porting my Python implementation of this approach to Ruby, in the Redux project on Rubyforge. Redux stands for 'Ruby Event-Driven Update Exchange'. It uses the highly scalable EventMachine epoll-based event loop to power its event-driven architecture. This will be essential when Redux is asked to scale up a Comet-based application.
Like Rails, Redux will be a Web (2.0) application framework, but unlike Rails, it puts the Observer Pattern and event- and rule-driven programming at its core.
Redux's headline is 'Web 2.0 in-a-box' or 'Naked Objects on the Web'.
Conclusion
If you're in BigCo, and are responsible for setting BigCo's technical strategy, then train your Java devs up on Web 2.0 core technologies such as Ajax, JSON, Atom, Microformats and OpenID.
And fire up their enthusiasm by tapping into Ruby (perhaps via JRuby) on your way to the Web 2.0 platform.
Learn patterns for mashing and integrating. Learn about REST and event- and rule-driven programming, including declarative DSLs.
When this Web platform hits BigCo, you will probably find that its REST or ROA style make your SOA integration strategy look rather complex and unweildy.
Check out the Distributed Observer Pattern, and download Redux when it's done (I'll let you know if you subscribe here!).
In 2007 and beyond, its the Web itself that's the platform, not Java or .Net. But if you want to get there via a language-based platform, Ruby could be the best way to transition to it.
Note: Everything I said about Ruby and Rails applies equally in technical terms to Python and Django, but regardless of the significant benefits of the latter, Ruby and Rails have the Web 2.0 market and mindshare. I'll probably switch this blog from Django to Redux sometime this year..
(c) 2007 Duncan Cragg
Open Data .. has .. recently .. been.. all .. over .. the .. blog .. o .. sphere!
Openness is a classic Us-and-Them issue. Big, nasty Apple/MySpace/Flickr is trying to control what little me/SingleStatus/Zoomr can do with my/our own stuff.
Open Data vs. Closed; Open Source vs. Proprietary; P2P vs. DRM; privacy vs. surveillance. The battles between the freedom of the pioneer, the individual and the minority against the rules and stability of the establishment and the majority form the endless shape of human history.
Us beating Them is Hollywood's favourite subject on-screen - and ironically Them fighting Us Hollywood's favourite battle off-screen.
As an Us-and-Them issue, with Us less powerful than Them, it's also tempting to give up and to follow the crowd - to do what we're told, to not ask for or sieze the privacy and open data we feel entitled to.
However, at XTech 2006 recently, there was a set of talks on the subject with a more positive approach. ...
Open Data .. has .. recently .. been.. all .. over .. the .. blog .. o .. sphere!
Openness is a classic Us-and-Them issue. Big, nasty Apple/MySpace/Flickr is trying to control what little me/SingleStatus/Zoomr can do with my/our own stuff.
Open Data vs. Closed; Open Source vs. Proprietary; P2P vs. DRM; privacy vs. surveillance. The battles between the freedom of the pioneer, the individual and the minority against the rules and stability of the establishment and the majority form the endless shape of human history.
Us beating Them is Hollywood's favourite subject on-screen - and ironically Them fighting Us Hollywood's favourite battle off-screen.
As an Us-and-Them issue, with Us less powerful than Them, it's also tempting to give up and to follow the crowd - to do what we're told, to not ask for or sieze the privacy and open data we feel entitled to.
However, at XTech 2006 recently, there was a set of talks on the subject with a more positive approach.
I'm counted amongst the 'founding members' of the UK's Open Rights Group, having signed the pledge in PledgeBank. So I was delighted to see Suw Charman's name on the XTech programme.
Suw gave a talk about the issues they/we are tackling: ISPs and Telcos tracking our traffic, DRM controlling our media and even our computers, ID cards, Patents, various government schemes to undermine our privacy, Copyright extension, Trusted Computing, etc., etc.
Read about the issues, then join the group (or the EFF, etc., depending on where you live).
Now, this was cool. I'll start right off by noting that they have a REST API, not a STREST API..
And it just gets better from there.. OpenStreetMap is like a Geo Wiki of GPS trails around the world. The centre of London was mapped by attaching GPS devices to couriers and plotting the slug-trails they left behind. Steve Coast showed an animation of a day's tracking. The trails shot off to various points out of town at the end of the day. But the animation continued..
And at pub closing time, another burst of trails as the couriers came home!
A few weeks ago, a crowd of OpenStreetMap volunteers descended on the Isle of Wight in England to walk around mapping it. The enthusiasm generated by this project seems likely to give it enough momentum that it will very soon be good enough for us to use for real. A complete, reliable dataset created by the people, for the people, unencumbered by copyright and restrictive licensing.
I suggested an idea I had about five years ago in my dot-com days: let people upload mobile-snapped pictures with GPS coordinates attached. Central servers scan the pictures for text and OCR them into a searchable database.
People would snap-and-upload their own street name signs, house name plaques, shop fronts, station name signs, company office signs, etc. Even menus in the local restaurant. We could construct a 'GeoGoogle' of public, mappable, searchable text.
However, Steve told me that OCR and mobile GPS technology still isn't up to it; that manual tagging of GPS trails is enough to make OpenStreetMap work well. Ah well, I've waited five years, so I can wait a bit longer...
Paul Hammond (ex-BBC, Yahoo!) gave a good talk on how to motivate companies to open up their data to the Mashable Data Web, hopefully through nice shiny new Web 2.0 APIs.
Here's your takeaway quote courtesy of Suw Charman's amazing typing:
- Be aware of the problems
- Demonstrate usefulness, screen scrape if you need to, but don't get yourself cease-and-desisted
- Don't assume it's a technology problem
- Target the right people, find someone on the inside who can help you
- Talk about benefits to the provider, not the consumer. If you talk about the benefits to you, they'll see you just as someone who wants something for free.
- Have patience. It is getting better every day, and it takes time for business to come round.
The Data Web will be created mostly from the edges, but the middle should be encouraged to see the benefits, too.
Native to a Web of Data: Designing a Part of the Aggregate Web
Yahoo! had a big presence at XTech - as did the BBC - and this Yahoo! - and ex-BBC - presenter was Tom Coates.
Tom's talk extended the previous one of his colleague, Paul Hammond, by going over some principles of opening data and helping build the Data Web.
Principles such as finding data sources, giving them nice representations and URLs then distributing them in a most accessible and navigable way.
A bit Web 1.0, in truth; he looked rather blankly at me when I suggested adding some concept of data events or updates to his list of data-opening principles.
Again, Suw has this written up (actually, a better-looking version of the same talk from FoWA).
Oh - here's a good article by Tom on the power of pointability
Harnessing the Collective
All of these talks had a positive spin on the Us-and-Them issue.
We can do something about the creeping imbalance of Digital Rights, tipping slowly but persistently from Us to Them.
Web 2.0 and the Social Software it underpins can actually empower those of us that are willing to use it.
As taxpayers, voters, shareholders, pensionholders, customers and employees, we really do own and control Them. We are just one Social Networking movement away from realising that - in both senses of the word.
So: stop staring at YouTube, sign up to your local Digital Rights organisation, get out on your bike with your GPS, understand and talk nicely to that unenlightened, data hoarding corporation and start a bottom-up revolution to take back control over your own data...
Above all, design your systems to put Open Data at the top of your list of priorities...
Declarative Architectures focus on the What, not the How, of programming. The How has dominated the field - perhaps 80% of programming is done in the traditional Imperative style, where we tell the computer How to do a task in explicit steps.
I'd like to show in this blog how Declarative Architectures and technologies are not just an interesting sideshow to the main Imperative attraction, but a complete and powerful programming alternative in their own right - indeed, one which has already dominated certain fields.
Imagine being able to simply express What we want the computer to do - to give it constraints and rules - then let it work out for itself How to achieve our goals.
I believe that saying What, not How, will become the dominant paradigm in programming. ...
Declarative Architectures focus on the What, not the How, of programming. The How has dominated the field - perhaps 80% of programming is done in the traditional Imperative style, where we tell the computer How to do a task in explicit steps.
I'd like to show in this blog how Declarative Architectures and technologies are not just an interesting sideshow to the main Imperative attraction, but a complete and powerful programming alternative in their own right - indeed, one which has already dominated certain fields.
Imagine being able to simply express What we want the computer to do - to give it constraints and rules - then let it work out for itself How to achieve our goals.
I believe that saying What, not How, will become the dominant paradigm in programming.
What?
Declarative Architectures are about data: documents, assertions, truths, rules, tables, structures, state, bits. Data storage, changes, constraints, transformation, sharing, querying, fetching, interaction, presentation, subscription, tagging.
In summary, we talk about stuff, changes to stuff and rules governing stuff. Not processes, actions, jobs or functions.
It's amazing how many interesting Declarative technologies are actually out there already - especially given that most programming is Imperative. They tend to go unnoticed; subservient to their Imperative masters.
Here are some examples from our daily work life (assuming you work in computing!):
What all these have in common is that they are purely data- and event-driven or oriented. Not a thread in sight (if used properly). No method calls. Just querying, fetching, subscribing, tagging, declaring truths and rules, making changes to stuff, etc.
On the 'Net, all the dominant technologies are actually Declarative, not Imperative:
Again, the same things are happening: querying, fetching, subscribing, tagging, declaring truths and rules, making changes to stuff.
Email (SMTP) is a great example of a Declarative Architecture, with its routing and transformation rules and self-addressed store-and-forward transmission. And, in fact, much the same applies to the rest of this list, where the Declarative aspects may include network querying and data events.
Imperative protocols such as RPC, RMI, CORBA, SOAP, etc. haven't had anywhere near the impact of their Declarative cousins.
Now, consider the 'Web 2.0' technologies:
This is a wonderful list of Declarative approaches (as long as we use REST APIs and talk about Resources, not SOAP ones talking about functions and jobs!).
So far, I've mentioned technologies that have attached themselves to the periphery of Imperative programming, where the actual behaviour is still driven procedurally, or How-like.
However, Declarative programming itself has a history at least as long as Imperative programming, and includes:
We can, if we choose, write 100% Declarative systems (even down to asynchronous or event-driven operating systems and hardware).
Wherefore?
In this blog, I'll be pointing out opportunities where we can benefit from the Declarative approach when it's ready.
Perhaps the biggest example of a Declarative Architecture that is ready to benefit us now is the Web. In contrast, the Imperative Service-Oriented Architecture model and so-called 'Web' Services are still weaning; still trying to get a life free of their huge standards consortia. REST, in its simplicity, ubiquity and straightforward usefulness is orders of magnitude more deployed than SOAP will likely ever be.
This blog is for discussing the common problems and solutions in Declarative technologies; seeing if ideas in one area can be used in another and seeing if we can distill their lessons and their approaches.
Ultimately, heading towards a unified, distributed Declarative programming technology landscape.
I hope you'll join me in my explorations into Declarative Architectures, in particular if you're interested in the currently most promising segment I mentioned: Web 2.0, and its application in Social Software.
Or the currently massive but little-mentioned segment: Peer-to-Peer, and its application to the disruption of, well you know, all those big technological and commercial hierarchies...