<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<?xml-stylesheet href="http://duncan-cragg.org/css/atom.css" type="text/css" ?>
<!-- Copyright (c) 2006 Duncan Cragg -->

<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-gb">
    <id>http://duncan-cragg.org/blog/</id>
    <title>What Not How</title>
    <subtitle>Duncan Cragg on Declarative Architectures</subtitle>
    <author><name>Duncan Cragg</name></author>
    <logo>/favicon.gif</logo>
    <icon>/favicon.ico</icon>
    <rights>All content including photos and images by Duncan Cragg. Copyright (c) Duncan Cragg, your rights preserved: see /CXL.html</rights>
    <generator uri="http://www.djangoproject.com">A Django Production.</generator>
    <link rel="alternate" type="text/html" href="http://duncan-cragg.org/blog/" title="What Not How" />
    <link rel="self" type="application/atom+xml" href="http://duncan-cragg.org/blog/atom/" />

    <updated>2010-03-18T16:58:00Z</updated>


    <entry>
        <id>http://duncan-cragg.org/blog/post/json-mash/</id>
        <title>JSON-Mash</title>
        <published>2010-03-18T16:58:00Z</published>
        
        <updated>2010-03-18T16:58:00Z</updated>
        
        <link rel="alternate" type="text/html" href="http://duncan-cragg.org/blog/post/json-mash/" title="JSON-Mash" />
        
        <category term="declarative" />
        
        <category term="event-driven" />
        
        <category term="rest" />
        
        <category term="json" />
        
        <category term="rest-observer" />
        
        <category term="forest" />
        
        <category term="fjord" />
        
        <summary type="xhtml">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>

Around the middle of February I completed a basic persistence and networking
implementation for
<a href="http://duncan-cragg.org/blog/post/fjord-in-memory/">Fjord</a>,
then had to do other things for a month.  Just recently I
fixed Fjord to work with the latest version of the 
<a href="http://nodejs.org/">Node.js</a> APIs.
</p><p>
Next project: I&#39;m going to use Fjord in a Web Framework to be called &quot;JSON-Mash&quot;.
</p><p>
I intend to show that JSON-Mash will be a great framework for rapidly building
truly interoperable and truly scalable online and distributed functionality.
</p><p>
Here&#39;s how JSON-Mash will work.
 &#160; ...
</p>

            </div>
        </summary>
        <content type="xhtml" xml:space="preserve">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>
</p><div class="summary"><p>
Around the middle of February I completed a basic persistence and networking
implementation for
<a href="http://duncan-cragg.org/blog/post/fjord-in-memory/">Fjord</a>,
then had to do other things for a month.  Just recently I
fixed Fjord to work with the latest version of the 
<a href="http://nodejs.org/">Node.js</a> APIs.
</p><p>
Next project: I&#39;m going to use Fjord in a Web Framework to be called &quot;JSON-Mash&quot;.
</p><p>
I intend to show that JSON-Mash will be a great framework for rapidly building
truly interoperable and truly scalable online and distributed functionality.
</p><p>
Here&#39;s how JSON-Mash will work.
</p></div><p>
<b>Juggling Data Formats</b>
</p><p>
In traditional Web Frameworks, the developer has to juggle up to five different data
formats. 
</p><p>
At the front end, you may be working in HTML or HTML chunks sent via Ajax. Then your
domain logic may use object classes. You&#39;ll no doubt map those objects to relations in
your database layer. That&#39;s three data representations (also known as
<a href="http://research.microsoft.com/en-us/um/people/emeijer/papers/xml2003/xml2003.html">Circles, Triangles and Rectangles</a>).
</p><p>
But there may be even more: if you call out to integration points such as a search
service or other API, you&#39;ll probably be using XML. And if you did application-level
cacheing, you would probably consider casting your data into key-value form.
</p><p>
This is not unrealistic - at a current ThoughtWorks client, I was faced with exactly
these five data formats - in an extremely simple application.
</p><p>&#160;</p><p>
<b>JSON Everywhere</b>
</p><p>
JSON-Mash will do away with all that. It uses JSON objects everywhere. 
</p><p>
JSON-Mash sends JSON data to a script in the browser that builds the page declaratively
from its understanding of a number of stable, generic JSON types. It uses Ajax and Comet
to pull JSON chunks in, to composite into the page.
</p><p>
JSON-Mash uses Fjord to process its JSON: transforms, map-reduce, etc. It uses Fjord to
persist objects in a JSON NoSQL database, and Fjord to provide shared JSON cacheing
functionality with cache updating via Fjord&#39;s
<a href="http://duncan-cragg.org/blog/post/deriving-forest/">FOREST</a>
integration style.
</p><p>
Finally, if there&#39;s any external system to be integrated, that too will be accessed via
an adaptor that converts its API, etc., into JSON-Mash stabilised JSON. There will be
Twitter and Flickr adaptors at first, I should imagine.
</p><p>&#160;</p><p>
<b>JSON-Mash and Fjord</b>
</p><p>
Using Fjord means that JSON-Mash treats its JSON as a first-class hyperdata type: lots
of little JSON chunks all with URLs and all linked up. I 
<a href="http://duncan-cragg.org/blog/post/introducing-fjord/">suggested</a>
that one day it may have the media type of &quot;<code>application/fjord+json</code>&quot;.  JSON-Mash Nodes
communicate in the 
<a href="http://duncan-cragg.org/blog/post/deriving-forest/">FOREST</a>
style.
</p><p>
Of course, the main benefit of using Fjord is that you get a great, declarative mashup
language to describe your logic in. 
</p><p>
Fjord is still very new, of course, so some goodness is yet to be rounded out. For
example, it comes with a very basic, but highly improvable, NoSQL JSON datastore and
already allows some powerful map-reduce and other query and transformation modes, with
more to come.
</p><p>
Finally, Fjord gives a dynamic view of JSON data: caches get updates pushed into them
and Comet pull will also come as part of the package.
</p><p>&#160;</p><p>
<b>CSI and SSI</b>
</p><p>
Now, one of the great things about Node.js is that you can run the same code on the
server as you run in the browser.
</p><p>
My previous attempts at doing things the JSON-Mash way have met with some resistance
when I suggested building duplicate logic server-side to render pages statically, for
accessibility and search engines. 
</p><p>
But with Node, this problem goes away: you can use the exact same Javascript to convert
JSON to HTML on the server side as you do on the browser.
</p><p>
I use the phrases &#39;Client-Side Include&#39; and &#39;Server-Side Include&#39; or CSI and SSI as
easily-understood shorthand for this kind of page assembly from JSON elements.
</p><p>
Settling on a number of stable JSON types means that - once the types and their
rendering are stable and coded - you never need to write any Javascript yourself in
JSON-Mash. It&#39;s all done declaratively, and looks the same whether generated by an SSI
Node or the CSI in the browser. For example, you can trigger calendars and maps in the
browser view, with just small JSON declarations.
</p><p>
This approach differs from &quot;progressive enhancement&quot; as long as you have both full CSI
and full SSI to render your site. But you could also have a &#39;CSI-SSI slider&#39;, where an
initial page is generated by SSI, then CSI takes over to fill in the funky stuff: the
dynamic widgets and Comet-driven elements.
</p><p>&#160;</p><p>
<b>Users Exploring a Dynamic Web of JSON</b>
</p><p>
The way JSON-Mash works is that you get a view into a hyperlinked Web of JSON data, and
can explore it from any point outwards. In the CSI version, each JSON chunk is loaded
lazily as it is encountered, then reloaded when it changes.  Even the SSI version of the
site would allow this form of exploration, although in a less efficient and dynamic way,
of course. 
</p><p>
In JSON-Mash, the user is also a first-class object; the stable JSON types also include
a type representing the user. This user is driving their browser, subscribing to events
on visible JSON page components via Comet, and POSTing their own user state to a
server-side cache copy.
</p><p>
Hence, if that user state contains a message, you can get simple chat functionality,
without any code, just by linking to a user and viewing them.
</p><p>&#160;</p><p>
<b>Conclusion</b>
</p><p>
JSON-Mash is a Web Framework that will use, re-use, assemble, mash, transform, store,
share, push, pull and render JSON from end to end and top to bottom; from Ajax to NoSQL.
</p><p>
Most domain-logic programming will be done in simple, powerful declarative functions,
with adaptors, drivers or renderers written in Javascript for Node.js or the browser.
Cross-host exchange is done in the FOREST style.
</p><p>
SSI or CSI page assembly are very flexible ways of mashing up new functionality from multiple
back-end sources of RESTfully-sourced, interlinked JSON data.  And with cache updates
telling SSI and CSI when the back-end data has changed, you get a very dynamic yet
scalable system.
</p><p>
JSON-Mash will be a framework for rapidly building truly interoperable and truly
scalable online and distributed functionality. 
</p><p>
I&#39;ll write more on the benefits once I&#39;ve got some code to prove it!  Follow me on 
<a href="http://twitter.com/duncancragg">Twitter</a>
and I&#39;ll let you know when I&#39;ve got something up on 
<a href="http://github.com/duncancragg">GitHub</a>.

</p>

            </div>
        </content>
    </entry>
    
    <entry>
        <id>http://duncan-cragg.org/blog/post/fjord-in-memory/</id>
        <title>Fjord in Memory</title>
        <published>2010-01-26T13:46:00Z</published>
        
        <updated>2010-01-26T13:46:00Z</updated>
        
        <link rel="alternate" type="text/html" href="http://duncan-cragg.org/blog/post/fjord-in-memory/" title="Fjord in Memory" />
        
        <category term="json" />
        
        <category term="forest" />
        
        <category term="fjord" />
        
        <summary type="xhtml">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>

Right, I&#39;m pleased to say that I&#39;ve now implemented enough of the 
<a href="http://duncan-cragg.org/blog/post/fjord-node/">Fjord language</a>
on <a href="http://nodejs.org/">Node.js</a> to be able to run the 
<a href="http://duncan-cragg.org/blog/post/introducing-fjord/">Instrument example</a>
that I introduced it with.  As yet, this runs in memory only - i.e., no disk, no
network.
</p><p>
<a href="http://github.com/DuncanCragg/Fjord/">Here&#39;s the code on GitHub</a> with tests
that show how it works. The language has changed a little so I&#39;ll show the example here
again, copied over from the test code, in order to explain the differences.
 &#160; ...
</p>

            </div>
        </summary>
        <content type="xhtml" xml:space="preserve">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>
</p><div class="summary"><p>
Right, I&#39;m pleased to say that I&#39;ve now implemented enough of the 
<a href="http://duncan-cragg.org/blog/post/fjord-node/">Fjord language</a>
on <a href="http://nodejs.org/">Node.js</a> to be able to run the 
<a href="http://duncan-cragg.org/blog/post/introducing-fjord/">Instrument example</a>
that I introduced it with.  As yet, this runs in memory only - i.e., no disk, no
network.
</p><p>
<a href="http://github.com/DuncanCragg/Fjord/">Here&#39;s the code on GitHub</a> with tests
that show how it works. The language has changed a little so I&#39;ll show the example here
again, copied over from the test code, in order to explain the differences.
</p></div><p>
Starting with the instrument rules:
</p><pre>
inrl1=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                       &#39;  &quot;%owid&quot;: &quot;/$this/&quot;,&#39;+
                       &#39;  &quot;%refs&quot;: { &quot;tags&quot;: [ &quot;equity&quot;, &quot;bid&quot; ],&#39;+
                       &#39;             &quot;%owid&quot;: &quot;/$bid/&quot;,&#39;+
                       &#39;             &quot;on&quot;: &quot;/$this/&quot; },&#39;+
                       &#39;  &quot;buyers&quot;: &quot;/array/has($bid)/&quot; }&#39;);
// -------------------------------------------------------------------
inrl2=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                       &#39;  &quot;%owid&quot;: &quot;/$this/&quot;,&#39;+
                       &#39;  &quot;%refs&quot;: { &quot;tags&quot;: [ &quot;equity&quot;, &quot;ask&quot; ],&#39;+
                       &#39;             &quot;%owid&quot;: &quot;/$ask/&quot;,&#39;+
                       &#39;             &quot;on&quot;: &quot;/$this/&quot; },&#39;+
                       &#39;  &quot;sellers&quot;: &quot;/array/has($ask)/&quot; }&#39;);
</pre><p>
These are the rules that add - or, rather, ensure the presence of - referring bids and
asks in the instrument&#39;s buyers and sellers lists. The big difference here is that
referrers are now accessed through the &#39;%refs&#39; tag. Both &#39;%refs&#39; and &#39;%owid&#39; are
available on all Fjord objects: %owid is the Object Web ID of the object, and %refs has
a list of the OWIDs of all currently referring objects.
</p><p>
But, of course, you can&#39;t see the list or the OWIDs under %refs! This rule shows how
Fjord can match single items to lists of items, and how it transparently jumps any OWIDs
it comes across, to continue matching. In this case there is a pattern for a bid or ask
object that refers to the instrument - notice the  &quot;on&quot;: &quot;/$this/&quot; match. 
</p><p>
Every object in %refs must have some such link to the referred-to object that triggered
the referral in the first place. This referrer becomes an observer: its rules are run
again if any object it refers to changes.
</p><p>
More instrument rules:
</p><pre>
inrl3=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                       &#39;  &quot;buyers&quot;:  { &quot;price&quot;: &quot;/$bids;number/&quot; },&#39;+
                       &#39;  &quot;bid-ask-spread&quot;: { &quot;high-bid&quot;: &quot;/number/max($bids)/&quot; } }&#39;);
// -------------------------------------------------------------------
inrl4=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                       &#39;  &quot;sellers&quot;: { &quot;price&quot;: &quot;/$asks;number/&quot; },&#39;+
                       &#39;  &quot;bid-ask-spread&quot;: { &quot;low-ask&quot;:  &quot;/number/min($asks)/&quot; } }&#39;);
</pre><p>
These are the rules that set the bid-ask-spread of the instrument from the prices of the
bids or asks in each list. It shows how, when a single match instance binds to a list,
it can give a match set in any bound variables - here, &#39;$bids&#39; and &#39;$asks&#39; - which can
then be reduced - here, using &#39;max()&#39; and &#39;min()&#39;.
</p><p>
The rules are now separated into two, as there was a historic <i>first ever bug in
public Fjord code</i>: the original rule wouldn&#39;t match unless at least one buyer <i>and</i>
one seller were present, and we&#39;re adding them one by one from scratch this time!
</p><p>
Here is the actual instrument object:
</p><pre>
inst=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                      &#39;  &quot;long-name&quot;: &quot;Acme Co., Inc&quot;,&#39;+
                      &#39;  &quot;buyers&quot;: [ ],&#39;+
                      &#39;  &quot;sellers&quot;: [ ],&#39;+
                      &#39;  &quot;bid-ask-spread&quot;: { &quot;high-bid&quot;: &quot;10.0&quot;, &quot;low-ask&quot;:  &quot;20.0&quot; } }&#39;,
                      [ inrl1, inrl2, inrl3, inrl4 ] );
</pre><p>
Notice the list of rules that the object will be animated by: inrl1, etc., which are the
rule object OWIDs, not the actual rule objects. The bid-ask-spread is set to some seed values.
</p><p>
On to bid and ask rules:
</p><pre>
bidrule=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;bid&quot; ],&#39;+
                         &#39;  &quot;on&quot;: { &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                         &#39;          &quot;bid-ask-spread&quot;: { &quot;high-bid&quot;: &quot;/$hibid;number/&quot; } },&#39;+
                         &#39;  &quot;price&quot;: &quot;/null/fix(2, $hibid * 1.10 )/&quot; }&#39;);
// -------------------------------------------------------------------
askrule=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;ask&quot; ],&#39;+
                         &#39;  &quot;on&quot;: { &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                         &#39;          &quot;bid-ask-spread&quot;: { &quot;low-ask&quot;: &quot;/$loask;number/&quot; } },&#39;+
                         &#39;  &quot;price&quot;: &quot;/null/fix(2, $loask * 0.90 )/&quot; }&#39;);
</pre><p>
Here we have rules for setting the price of a bid or an ask as a ratio of the
instrument&#39;s bid-ask-spread numbers. These are not meant to be realistic business rules
of course - they&#39;re for illustration purposes. The only real difference here is fixing
the result to 2 decimal places, since Javascript was adding ugly rounding errors.
</p><p>
Now we go ahead and create two bids and two asks, pointing at the instrument by simply 
including its OWID:
</p><pre>
bid1=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;bid&quot; ], &quot;on&quot;: &quot;&#39;+inst+&#39;&quot;, &quot;price&quot;: &quot;&quot; }&#39;, [ bidrule ]);
bid2=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;bid&quot; ], &quot;on&quot;: &quot;&#39;+inst+&#39;&quot;, &quot;price&quot;: &quot;&quot; }&#39;, [ bidrule ]);
ask1=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;ask&quot; ], &quot;on&quot;: &quot;&#39;+inst+&#39;&quot;, &quot;price&quot;: &quot;&quot; }&#39;, [ askrule ]);
ask2=WebObject.create(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;ask&quot; ], &quot;on&quot;: &quot;&#39;+inst+&#39;&quot;, &quot;price&quot;: &quot;&quot; }&#39;, [ askrule ]);
</pre><p>
This fires off all the rules and notifications .. and rules and notifications .. giving
the final result:
</p><pre>
exp=new WebObject(&#39;{ &quot;tags&quot;: [ &quot;equity&quot;, &quot;instrument&quot; ],&#39;+
                  &#39;  &quot;long-name&quot;: &quot;Acme Co., Inc&quot;,&#39;+
                  &#39;  &quot;buyers&quot;:  [ &quot;&#39;+bid1+&#39;&quot;, &quot;&#39;+bid2+&#39;&quot; ],&#39;+
                  &#39;  &quot;sellers&quot;: [ &quot;&#39;+ask1+&#39;&quot;, &quot;&#39;+ask2+&#39;&quot; ],&#39;+
                  &#39;  &quot;bid-ask-spread&quot;: { &quot;high-bid&quot;: &quot;12.1&quot;, &quot;low-ask&quot;:  &quot;16.2&quot; } }&#39;)
// -------------------------------------------------------------------
test.objectsEqual(&quot;Instrument example works&quot;, Cache[inst], exp);
</pre><p>
Here the buyers and sellers correctly list the referring bids and asks, and the
bid-ask-spread has been set to the values you get after two ratios are applied to the
original seed numbers.
</p><p>
Rewrite rules are run on object construction, they&#39;re run whenever an object has a new
referrer in its &#39;refs&#39; list, and also whenever another object that&#39;s being referred to
changes. The latter isn&#39;t exercised in this example, as all events are propagated by
referral alone. The tests of Fjord have examples of the full Observer Pattern in operation.
</p><p>&#160;</p><p>
<b>Next Up</b>
</p><p>
Now, this is living code, so it will no doubt be different again, soon. Watch this space, and
<a href="http://twitter.com/duncancragg">follow me on Twitter</a>, to keep up to date with developments.
</p><p>
Next, I&#39;ll be working on simple disk persistence of Fjord JSON objects and rules,
followed by distribution in the 
<a href="http://duncan-cragg.org/blog/post/deriving-forest/">FOREST</a> style...

</p>

            </div>
        </content>
    </entry>
    
    <entry>
        <id>http://duncan-cragg.org/blog/post/fjord-node/</id>
        <title>Fjord in Node</title>
        <published>2010-01-06T17:03:00Z</published>
        
        <updated>2010-01-06T17:03:00Z</updated>
        
        <link rel="alternate" type="text/html" href="http://duncan-cragg.org/blog/post/fjord-node/" title="Fjord in Node" />
        
        <category term="event-driven" />
        
        <category term="json" />
        
        <category term="forest" />
        
        <category term="fjord" />
        
        <summary type="xhtml">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>

Well, I&#39;ve put together the first few lines of <a href="http://duncan-cragg.org/blog/post/introducing-fjord/">Fjord</a>, implemented on <a href="http://nodejs.org/">Node.js</a>.
</p><p>
Here&#39;s the description on <a href="http://github.com/DuncanCragg/Fjord">GitHub</a>: <i>Fjord is a language for expressing domain logic as match-rewrite functions over mashable JSON Web objects</i>.
</p><p>
I&#39;m developing Fjord very openly, in the hope someone out there will be interested in getting involved in helping guide its design and implementation. I suppose code speaks louder than blog posts.
 &#160; ...
</p>

            </div>
        </summary>
        <content type="xhtml" xml:space="preserve">
            <div xmlns="http://www.w3.org/1999/xhtml">

<p>
</p><div class="summary"><p>
Well, I&#39;ve put together the first few lines of <a href="http://duncan-cragg.org/blog/post/introducing-fjord/">Fjord</a>, implemented on <a href="http://nodejs.org/">Node.js</a>.
</p><p>
Here&#39;s the description on <a href="http://github.com/DuncanCragg/Fjord">GitHub</a>: <i>Fjord is a language for expressing domain logic as match-rewrite functions over mashable JSON Web objects</i>.
</p><p>
I&#39;m developing Fjord very openly, in the hope someone out there will be interested in getting involved in helping guide its design and implementation. I suppose code speaks louder than blog posts.
</p></div><p>
It&#39;s going to be slow progress, with a wife, two kids, and Thoughtworks all together getting 99% of my life, but at least people will be able to keep up with what I&#39;m trying to do... I only got the project started at all because I&#39;m off sick right now!
</p><p>
Here&#39;s the GitHub link: <a href="http://github.com/DuncanCragg/Fjord">http://github.com/DuncanCragg/Fjord</a>.
</p><p>
Follow me on Twitter if you want all the latest news .. <a href="http://twitter.com/duncancragg">http://twitter.com/duncancragg</a>
</p><p>
Oh - I was lying about FOREST being GET-only, that was just to engage the REST community, but since I got zero reaction from anyone I may as well admit that I intended FOREST to use POST all along - to push updated resources into subscriber caches as I have always done. Or to use long-GETs to pull them in. Both are tunnelling cache refreshes over HTTP. Neither are Official REST.
</p><p>
Cheers!

</p>

            </div>
        </content>
    </entry>
    
</feed>

