|About All Things...|
Publish / Subscribe
|...taking programming beyond:|
Threads, Message Queues, Client-Server, CORBA, Web Services, SOAs, Agents, Synchronous Architectures, Imperative Programming - and even Applications, Desktops and Documents
|...works for ThoughtWorks UK; originally from April 2002 to July 2007 and now recently re-joined. Previously worked as a Web Architect for the Financial Times.|
|...went to both UCL and Imperial College of the University of London (in the Eighties); specialising in Logic during his MSc.|
|...wonders when his LinkedIn Account will be useful|
|...has a phone-cam, and used it on himself once, just before his weekly shave:|
|...can be contacted by
and followed on Twitter.
May 19, 2014 21:21
With the expansion of the Internet of Things (IoT), and the diversity of products and
technologies, the one thing that everyone agrees on is that it's time to start agreeing:
the Internet of Things needs standards. Many agree that it needs open standards, like
those that underpin the Web.
Obviously, a Web of Things is going to be quite different from the Web of Documents and
Applications: it'll be much more fine-grained and much more "buzzy", with many sensors
and actuators working together with many hubs and services. It's more likely to be at
home with the next generation of the Internet Protocol: IPv6.
To meet the fine-grained and buzzy nature of the IoT, the
Constrained Application Protocol, or CoAP,
was created. CoAP is an open Internet standard for the Web of Things. It's based
on the Web's core pipe: HTTP, but has many differences to allow it to be used by very
resource-constrained devices and local radio networks.
CoAP can be used in many different ways, but there's a danger that a lack of clarity in
exactly how it's used means it doesn't achieve its full potential to link up the world's
This article proposes a simple and clear way that CoAP could be used to build a uniform,
global, decentralised Web of interacting and discoverable Things.
This article first appeared on the ThoughtWorks Insights pages.
CoAP's Interaction Model
In simple terms, CoAP says you should send an Internet packet (UDP) to request a device's
data - a GET on a device URL - and then expect back a packet with that data, perhaps a
sensor value. You can also push a packet of data to a device - a POST to its URL.
An extension to CoAP for observation
allows you to keep on receiving back data packets following a GET as the device state at
its URL changes.
Ways of Interacting and Programming
This simple protocol is actually quite unique in the IoT world. Most IoT approaches work
through events and messages - sensors and actuators may be connected to an event stream
or message bus. The model used to program Things reflects this: events and messages tend
to lead to actions and commands, and thence to central control, either in a central
server in the house or a service in the cloud. Central control tends towards silos and
proprietary formats and application protocols.
CoAP's Web-based model of devices with their own URLs is quite different. The Web that
CoAP belongs to works through decentralisation - empowering everyone to publish and link
up to anyone else. These links encourage everyone to use the same formats for their
data, leading to re-use and "mashability". The lack of centralisation and greater
interoperability leads to greater freedom to quickly innovate and build on top of each
But since the event/action or message/command model is so familiar in the IoT world,
many people will inevitably try to use CoAP in this way, and miss out on all the
benefits that have been proven by the unprecedented scale of the Web.
And even if they do embrace the CoAP and Web model, there's still sufficient flexibility
in there to allow various groups of people to go their own way and lose many of the
benefits. For example, people still don't have to use links between their worlds and
don't have to share the same formats. They don't have to use POST in exactly the
same way to achieve dynamic interaction with devices. The way Web APIs have gone - to
create thousands of isolated silos -
is strong evidence of this risk.
Peer-to-Peer Programming of Things
So what way should everyone use CoAP? Well, like I said, people should at least link
up in a Web and use the same formats for lights and dimmers and locks and temperature.
And use the definition of those formats to guide the POST-side interaction models.
However, unlike the Web of large-scale documents and applications, of browsers and
webservers and client-server asymmetry, the Internet of Things is an Internet of
sensors and actuators and hubs; interacting devices which work as symmetric peers in
a, potentially IPv6, mesh.
This symmetry and peer-to-peer interaction are embraced in the CoAP model and can lead
to an interesting, simple and powerful approach to the Internet of Things, and to the
evolution of the Web itself. There's an opportunity here to go one step further than the
Web, through a simple and powerful programming model based on peer observation.
The Light Depends On The Light Level Sensor
For example, consider possibly the simplest IoT interaction imaginable: a light that
reacts to a sensor measuring the ambient light level and colour temperature, by
adjusting its brightness and colour accordingly, either to match or to compensate.
There are two peers - and we don't need a central controller. The light depends on the
sensor. They both have URLs, so the light can link to the sensor. We are using CoAP,
so the light can observe the sensor itself:
The light depends on the light level sensor. That's the simple, powerful interaction and
programming model that CoAP allows us to use - if we allow Things themselves to be the
observing client. The state of a Thing depends on the states of the other Things that it
links to. You can also have other abstract, non-real Things interacting in the peer
mesh, which we can still call Objects.
I won't go into detail here about exactly how CoAP would be used to achieve this: the
URLs and payload formats and the use of GET, observe and POST that enables Things to be
client observers. I'll document that elsewhere, but it'll be based on my
FOREST architectural style and the Object Network.
Things Watching Things
The Internet of Things can be a mashable Web of Things. If we're going to use CoAP to
build that fine-grained, "buzzy", decentralised Web, then we should use it like the Web:
we should agree on the formats for device data, and link up all of our Things and
non-Thing Object data via their URLs. The common formats should also define how POST is
to be used. That's the basic requirement.
We can then take a bold step further by exploiting CoAP's observation mechanism, to
build a symmetric peer mesh of Things and Objects that link to, watch and depend on each
others state: simply by allowing Things and Objects themselves to be the observing clients.
That's all I'm proposing - that Things just watch each other instead of being observed
by invisible processes.
This is a peer-to-peer model, where every player in the IoT is autonomous: empowered to
control itself and to evolve its own published state independently and concurrently.
That can create a global, visible, loosely-coupled peer mesh of mutually-observing,
interoperating and co-operating Things and Objects.
Programs to animate these peer Things and Objects can be extremely simple: you only need
to describe how their new state should be set as a function of the states around. This is
similar to programming a spreadsheet. It's also an inherently concurrent and distributable
programming model. My Cyrus programming language is designed to
express this kind of program.
March 1, 2014 11:24
After filling up that other
blog recently with 61
pages of content, one page a day, I was challenged by my ThoughtWorks colleague,
Andy McWilliams, to help him get in more
easily to my explanations of the Object Network applied to Augmented Reality and the
Internet of Things, especially around how my approach differs and is better than other
January 27, 2014 11:36
I've started another blog called Building The Object Network, about how I'm experimenting with Augmented Reality for the Internet of Things using the Object Network approach.
So far I've been blogging every day.
January 16, 2013 17:09
Well that worked out pretty well:
I have a 3D environment on Android programmed in a simple but powerful declarative
language which I've called "Cyrus".
Cyrus basically uses JSON all the way through: from user
interface and scene graph to rewrite rules, on the wire and on disk. The Cyrus
programming language is essentially JSON itself, as JSON rewrite rules. I've reduced the
noise of JSON in Cyrus by taking out redundant double-quotes, square brackets and
commas. It looks very nice to me.
August 17, 2012 11:11
If you also think that hacking up 3D worlds on Android could be fun, then join me! Stuff you should expect to play with if you want to get involved includes Java, Android, OpenGL ES 2.0, 3D model creation, hyperlinked JSON and JSON rewrite rules. Creatives, evangelists and inspirers are also very welcome to get involved!
The idea is to make an app (NetMash) that lets people build, mash up, animate and program 3D worlds, shared online and all linked-up, Web-like.
Like creative-mode Minecraft, but adding easy in-world programming and shared online by default. Or maybe a bit like an open, distributed, generic, mobile
here), for adults as well as children.
NetMash is intended to deliver creative empowerment to ordinary people. We professional software folk often get stereotyped as geeks, and the creative fun we often have dismissed as in some way unusual. That's a real shame, because such prejudice means that the other 99.9% of the world are simply missing out on the joy of experiencing the most creative and empowering activities humankind has yet invented.
August 15, 2012 11:37
I just re-read my article on the Universe Web. I think it's pretty good. Indeed, to be honest, "programming as Cyberspace building" is where my heart has always been, and I'm all about following my heart this year. Especially if it's more fun, for both myself and others! Or if it opens up new worlds to new people.
In contrast, I don't see "fun" in W3C or IETF activities. Indeed, there's recently been a number of examples of tension in that world, between stabilisation and innovation, idealism and pragmatism, Enterprisey and Webby. Interestingly, all those examples have a "2.0" flavour: HTML5 (Web 2.0), HTTP 2.0 and OAuth 2.0.
My own interests are rough consensus and running code; innovation and pragmatism. Webbiness not in the W3C sense - "Web" Services, Semantic "Web", "Web" Sockets, etc. - but in the sense of "the simplest thing that works". Which is the Web of HTTP (1.1), URLs, JSON and REST, or specifically my FOREST interpretation.
I crave the simple and powerful, the cool and the fun. Which ultimately leads to the kind of thing I was describing as the Universe Web. And to be honest, I'd like to write and code for me, not for my peers and colleagues or for my career.
So, to the pursuit of pure joy in place of compromise, I'll now be focusing my energies on the journey of evolving the NetMash Java server and Android app towards an online, open, hyperlinked virtual world that is programmable in-world by users using simple rules.
January 20, 2012 18:07
Updated: January 23, 2012 19:45
Right, let's get started with some basic conventions in the Object Network!
This part in the Object Network series
will cover URLs, HTTP headers and some common JSON patterns.
Updated 23/1/12: I changed the URLs in the example to have one of each type.
January 19, 2012 20:58
Updated: January 22, 2012 16:05
OK, I'm trying to take a Big Idea and make it as Simple As Possible to grasp.
If we link our JSON data together and use the same formats, then our mobile, browser and
server apps can become much simpler - through clean, stable, common, shared, re-used
code - and much more powerful - through clean, stable, common, shared, linked, cached data.
This is the second part in the
Object Network series,
which will guide you away from building isolated Web APIs to engaging in a linked-up data landscape.
November 29, 2011 23:11
Updated: January 22, 2012 16:03
It's interesting to compare the
current growth of Web APIs
with the early growth of the Web itself.
To save you jumping those links: the Web dramatically beats the APIs.
I believe that the most likely cause of such relatively slow growth (in what should be a
booming ecosystem) is that each API forms a closed silo and cannot benefit from any
network effects. Every API is different and there are no links between them. There
usually aren't any links within a silo. You can't even use a given API without first
consulting the documentation.
The Object Network is designed to fix this, with linked-up
JSON in common formats. This will allow easier mashing, sharing and cacheing of data and
allow client code to be shared and reused.
May 10, 2011 11:11
The Web, in its purest form - declarative HTML and CSS documents, XML feeds - is
mashable, linkable, sharable. It's easy to create documents that slot into the global
Web and can be accessed on any device; accessed by just a simple link. Servers can
easily scale through statelessness and cacheing.
Native Mobile Apps are fast and slick. They are intimate with the dynamic, interactive,
tactile mobile user interface, intimate with the capabilities of the device and intimate
with the domain of mobile: photos, locations, contacts, messages.
OTS is a simple, clean, powerful approach to delivering Mobile functionality and
content that is designed to realise these benefits of both Native Apps and the Web.
May 6, 2011 11:11
Like Subbu, I also have been sitting on a blog post
about the Richardson Maturity Model.
I have different reasons for feeling uncomfortable with this Model, however.
The following came out of a discussion on an internal list at ThoughtWorks, where a
number of people were talking about how they aspired to reach the "Holy Grail" of REST
Level 3, and still thought they were basically "doing REST" by addressing most of the
But, as indeed pointed out in that article, REST is only at Level 3.
However, fortunately, you can jump right to Level 3 without much effort.
May 5, 2011 12:16
This post is a response to a question that came up on an internal ThoughtWorks list.
The question was, in summary: "Is using JSON more RESTful than minting our own
Media Types as required, given that using raw JSON means reading inside the content in
order to know what type is being transferred?"
TL;DR: Yes, use a common Media Type and "switch" on the internal data type; create a
new Media Type only when something generic and broad and new and useful settles out.
Seems controversial to you? Read on...
February 12, 2011 14:07
Here's a quick catch-up of developments in FOREST.
I've been working much more on FOREST than Fjord or JSON-Mash recently, and it's coming along very nicely.
Actually, I've been lying flat out in bed with Pneumonia (and various other consequent ailments) for several weeks, so have been quietly tip-tapping away on a laptop resting on my tum, when I've had the energy.
Which has given me a chance to tidy up and finish off some FOREST stuff that I started last year...
March 18, 2010 16:58
Around the middle of February I completed a basic persistence and networking
then had to do other things for a month. Just recently I
fixed Fjord to work with the latest version of the
Next project: I'm going to use Fjord in a Web Framework to be called "JSON-Mash".
I intend to show that JSON-Mash will be a great framework for rapidly building
truly interoperable and truly scalable online and distributed functionality.
Here's how JSON-Mash will work.
January 26, 2010 13:46
Right, I'm pleased to say that I've now implemented enough of the
on Node.js to be able to run the
that I introduced it with. As yet, this runs in memory only - i.e., no disk, no
Here's the code on GitHub with tests
that show how it works. The language has changed a little so I'll show the example here
again, copied over from the test code, in order to explain the differences.