It's going to be slow progress, with a wife, two kids, and Thoughtworks all together getting 99% of my life, but at least people will be able to keep up with what I'm trying to do... I only got the project started at all because I'm off sick right now!

Here's the GitHub link: http://github.com/DuncanCragg/Fjord.

Follow me on Twitter if you want all the latest news .. http://twitter.com/duncancragg

Oh - I was lying about FOREST being GET-only, that was just to engage the REST community, but since I got zero reaction from anyone I may as well admit that I intended FOREST to use POST all along - to push updated resources into subscriber caches as I have always done. Or to use long-GETs to pull them in. Both are tunnelling cache refreshes over HTTP. Neither are Official REST.

Cheers!

Instrument, Bids, Asks

Let's take the simplest possible view of a financial market.

There's an Instrument, and there are Bids and Asks on that Instrument. The Instrument summarises all the Bids and Asks into the 'bid-ask spread', taking the highest bid price and the lowest ask price. Forget volumes, trades and any other detail.

The Instruments, Bids and Asks are on various servers across various organisations. That's the 'symmetric REST' part.

Here's a JSON Instrument resource at the URL shown. It holds three Bid or Ask objects:

http://the-bank.com/fjord/equity-instrument-23ab33-2319f.json
{
    "tags": [ "equity", "instrument" ],
    "long-name": "Acme Co., Inc",
    "buyers": [ 
        "@http://a-bank.com/fjord/equity-bid-9ac0d1-88ce1.json"
    ],
    "sellers": [ 
        "@http://c-bank.com/fjord/equity-ask-510efb-cca62.json",
        "@http://d-bank.com/fjord/equity-ask-8560ae-33eff.json"
    ],
    "bid-ask-spread": {
        "high-bid": "10.00",
        "low-ask":  "14.00"
    }
}

The '@' character is the flag that this string is a link or a URL. The meaningful names in those links are really just for humans and the layers below us - Fjord itself won't let you see inside them.

So now that we've introduced a JSON extension making it a hypermedia or hyperdata type, perhaps we should announce this convention in a media type string. How about "application/fjord+json"? This media type will also serve to set expectations on everything else about Fjord JSON objects, once the notation is stable.

Notice that, instead of 'class' or 'type' or a top-level tag, we have a list of tags. In Fjord, class is in the eye of the beholder - but, of course, must be a stable grammar in the eye of the generator!

Right, so here is a new Bid object held at 'B' Bank, that the Instrument hasn't seen yet:

http://b-bank.com/fjord/equity-bid-0043ee-419ac.json
{
    "tags": [ "equity", "bid" ],
    "on": "@http://the-bank.com/fjord/equity-instrument-23ab33-2319f.json",
    "price": "",
    "by": "@http://b-bank.com/fjord/hcard-b-bank-4990cd-232b0.json"
}

It has a pointer to the Instrument it is a bid on, a bid price that isn't yet set, and a link to the hCard object of the bank offering the bid and serving this Bid object.

So Bids and Asks can see Instruments and vice-versa, and each can set their own state accordingly, as part of a FOREST interaction.

The 'animation' of Bid, Ask and Instrument JSON object resources is driven by three simple Fjord rules...

Rule One: Bid Sets Price From Instrument

Here is a rule setting the price of a new Bid to be 10% more than the highest current Bid on the instrument, hopefully ensuring this Bid is now the high bid:

{
    "tags": [ "equity", "bid" ],
    "on": { "tags": [ "equity", "instrument" ],
            "bid-ask-spread": { "high-bid": "/$hibid;decimal/" }
          },
    "price": "/null/( $hibid * 1.10 )/"
}

This is a template that matches the Bid object and its dependencies and rewrites part of the Bid.

Notice how the Bid object's link to the Instrument object is jumped transparently: the Bid can 'see' right into the Instrument in order to do its matching. However, it cannot itself rewrite the Instrument - only Instrument rules can do that.

In Fjord, matching is written with the '/ ; /' notation: each part of this semicolon-separated list must match. In this case, the high bid has to bind to the identifier '$hibid' and has to be a decimal. The price must not be set yet.

The keys 'decimal' and 'null' are, of course, special reserved words for matching in patterns. The 'null' matches empty strings, lists and hashes.

The rewrite part follows at the end of a match and is also wrapped in slashes. Here, the price is being set to the high-bid plus 10%. The parentheses introduce an expression.

This rule only matches if the price is empty - once set, the rule will not refire.

For more complex rewrites, the following forms can also be used:

    "price": { "/null//": "( $hibid * 1.10 )" }
    "price": { "when": "/null/", "then": "( $hibid * 1.10 )" }

which allow full hashes and lists to be both matched and set.

Rule Two: Instrument Puts New Bid into its Buyers List

The Instrument adds the new Bid to its buyers list with this rule:

{   "#url": "/$bid/",
    "tags": [ "equity", "bid" ],
    "on": { "#url": "/this/",
            "tags": [ "equity", "instrument" ],
            "buyers": "/list/has($bid)/",
    }
}

This is a rule on the Instrument, that 'looks up to' and matches the enclosing Bid.

The '#url' is an optional, reserved-word entry that can only appear at the top of a JSON resource and holds that resource's URL. Here, it is bound to '$bid', picking up the URL of the Bid object that refers to and encloses this Instrument.

Further down, the "#url" reserved-word value of "this" indicates the start of the object that this rule applies to. If "this" is absent, then, of course, this is a rule on the top-level object, as in the first rule above.

The rewrite part matches the buyers list of the Instrument and applies the 'has()' operator which ensures the list now contains the wrapping Bid; it rewrites the list to include its argument if it doesn't already.

If this rule matches two wrapping Bids simultaneously, then '$bid' is set to a 'match set' - a simultaneous binding to the URLs of all of those Bids, and 'has()' must ensure that all of these Bids are now in the list at once.

If there is no change after a rule is applied, the resource's Etag isn't incremented, and dependent rules won't fire. Fjord may re-apply any rule at any time, usually on an incoming state change indicated by Etag, so matches have to be designed accordingly.

That's why we say 'has', not 'add', which would add the Bid again and again each time the rule is applied. We could say "/list;hasNo($bid)/add($bid)/", but that's uglier and more imperative than declarative.

Rule Three: Instrument Sets Bid-Ask Spread from Bids and Asks

This is what the instrument looks like, now, with the new link to our Bid:

{
    "tags": [ "equity", "instrument" ],
    "long-name": "Acme Co., Inc",
    "buyers": [ 
        "@http://a-bank.com/fjord/equity-bid-9ac0d1-88ce1.json",
        "@http://b-bank.com/fjord/equity-bid-0043ee-419ac.json"
    ],
    "sellers": [ 
        "@http://c-bank.com/fjord/equity-ask-510efb-cca62.json",
        "@http://d-bank.com/fjord/equity-ask-8560ae-33eff.json"
    ],
    "bid-ask-spread": {
        "high-bid": "10.00",
        "low-ask":  "14.00"
    }
}

And here is what the new B-Bank Bid looks like with its competitive price:

{
    "tags": [ "equity", "bid" ],
    "on": "@http://the-bank.com/fjord/equity-instrument-23ab33-2319f.json",
    "price": "11.00",
    "by": "@http://b-bank.com/fjord/hcard-b-bank-4990cd-232b0.json"
}

So now the Instrument has to adjust its bid-ask spread to accomodate the new Bid. This happens when the following rule matches, which it always does - but only fires when a change is detected:

{
    "tags": [ "equity", "instrument" ],
    "buyers":  { "price": "/$bids;decimal/" },
    "sellers": { "price": "/$asks;decimal/" },
    "bid-ask-spread": {
        "high-bid": "/decimal/max($bids)/",
        "low-ask":  "/decimal/min($asks)/"
    }
}

This rule matches as many times as there are Bids and Asks, meaning that the local identifiers $bids and $asks are each bound to match sets.

Notice how the '{ price .. }' patterns match elements of a list, in the same way an XPath can match either one or several elements in a sequence of XML elements.

The 'max()' and 'min()' operators take the highest and lowest values in a match set (or in a physical list, come to that). Once again, this rewrite means 'set this value and increment the Etag, but only if there's a change'.

Here, the high-bid becomes our new Bid's '11.00'.

Fjord

So that's a quick intro to some of my ideas for JSON matching and rewriting in a FOREST architecture.

Fjord is a declarative, JSON-encoded language for expressing how the state of a JSON resource is a function of both its own state and the state of peer JSON resources that it can see via links in a hyperdata Web.

Fjord objects are 'masters of their own destiny', not allowing other objects to tell them what to do except via declarative intention.

Further articles will give more proposals for the syntax and semantics of Fjord, including true unification, and put it into the set of languages that are ideal for programming parallel and distributed systems, as well as for integration.

Since Fjord is very young and I'm still implementing it (in Java), I would certainly welcome comments about it, or about the FOREST style that it supports. You can chat with me on Twitter, too!