The Distributed Web

Jeni — Sun, 11 May 2008 22:07:29 +0100

XTech was subtitled “the mobile web”, but one of the major themes for me was that of the distributed web. The first keynote, by Simon Wardley, gave a vision of a future in which hardware, frameworks and applications are services in the cloud rather than products on machines we own: where we use flickr to store our photographs, Google App Engine to host our applications, and Amazon S3 to store our data. In David Recordon’s keynote (written up by Jeremy Keith), he talked about small, specific services provided by sites that aren’t “destination sites”. The same picture was painted by Gareth Rushgrove in his talk on Design Strategies for a Distributed Web.

So I was surprised at how contentious Steven Pemberton’s talk on Why you should have a Website (thankfully again documented by Jeremy Keith) proved to be. Because to me it seemed to be the logical extension to the distribution of hardware, frameworks and application: the distribution of data. In fact, I’ve written about the same idea myself, as has Leigh Dodds, more recently.

From the session, the main question seems to be “how could we do flickr without them holding our data?” I don’t want to particularly pick on flickr, especially because it’s not one of the worst offenders, but the problem of serving and sharing images does illustrate a whole range of issues, so I will use it as an example. I could just as easily be talking about ancestry.com. The way I see it, you need three levels:

providers which make information available in known formats
user interfaces which provide the end-user with a way to access and manipulate the information
brokers which locate information on the web and provide an aggregated interface

(It occurs to me that this is similar to a model/view/controller architecture: the providers give the model, the user interfaces give the views and the brokers control the flow between the two.)

Where flickr is at the moment is a conglomeration of the three: to have your photo appear on flickr, and to gain the advantages that it gives you in terms of tag-based aggregations and social networking, you have to upload it. They are then the provider of the image+metadata (perhaps the only place it is located on the web), the user interface on the image+metadata (the interface through which the image is annotated), and the broker (they provide keyword-based retrieval, for example).

What would it look like to separate those functions?

First, you, as the owner of the image+metadata, could put your data anywhere: on a home wireless network box, on a webserver hosted by an ISP of your choice, on a site specifically designed for hosting photos. Your data is exposed to the larger web through a standard read/write protocol (I’m betting on AtomPub) that allows you to provide metadata both about resources and the links between resources. The point of it being read/write is that it allows other people to add metadata to or links from your resource to others, such as adding a comment on your image.

Second, an information broker would locate your photos by crawling for them (or perhaps by you submitting the URL somewhere, but mostly that shouldn’t be necessary). There are already information brokers around: Google provides a RESTful API for general search results, as does Yahoo!; at XTech, Richard Cyganiak talked about Sindice, and Aidan Hogan about the Semantic Web Search Engine, both of which crawl for RDF triples and provide an API for querying the results. In an AtomPub-based environment, you’d want an information broker that located Atom feeds and resources, indexed them, and provided an AtomPub-based API for publishers to use.

Third, a user interface would provide an attractive and usable front-end that brought together many different sets of information. For example, flickr might combine your friends feed with an image search to provide a view of images recently made available by your friends. There’s no requirement for your friends to use flickr for this to work: flickr queries a broker for a list of your friends, then queries a broker for images by a particular person, the broker searches its index and points the application to the original resources that are provided by your friends.

A user interface has another role, though: to add to the web. Flickr wants to make it easy to add tags to photos, to create sets and collections that help you navigate your photos, for others to add comments and so on and on. And that’s fine, because AtomPub is a read/write API. To add a tag to a photo, flickr simply edits the resource with PUT. To add a comment, it locates the comment feed (which would be referenced from the entry for the particular image) and POSTs to create a new resource. And everyone can see those changes — the added value that you get from a social network.

None of this is to say that a single application can’t act as provider, broker and publisher at the same time, but I’m certain that users will favour those applications that do all of each role: provide to the whole web, broker the whole web, provide a user interface to the whole web. Flickr is almost there, but it doesn’t do the whole brokering job because it only brokers the data it provides, and therefore it doesn’t provide the whole user interface job.

This distributed web is a clear win, particularly for users, over walled gardens. They can switch from user interface to user interface, even use more than one at a time (perhaps one application is good for browsing while another is good for categorising), without any cost. They can choose who to use to serve their information on the basis of things that matter when you’re serving information (low downtime, backups, security, etc.) rather than on how pretty an interface looks or how much functionality it gives you. On the other side of the equation, applications get to do one thing and do it well.

It seems to me that this is simply how the web works, and the questions we should be asking are about privacy and trust and licensing and revenue models and standards development.

XTech 2008

Jeni — Sun, 11 May 2008 11:25:40 +0100

I finally have some time to write about XTech. What a great conference! I know that Edd would like it bigger, but its modest size gives it a family feel. Like a family gathering, there are pontificating oldsters whose wisdom goes largely unappreciated by young upstarts who themselves bring energy and innovation to the crowd. And a bunch in the middle trying to translate across the gap: to explain the vision to the old and the reality to the new.

Another way of putting it is the divide between the XML crowd and the Web 2.0 crowd. In Sean McGrath’s closing keynote, thankfully written up by Jeremy Keith, he talked about navigating the path between the document web and the programmable web, and the danger of tipping the balance too much in either one way or the other. XTech provides a great service in providing that balance, and of giving those of us with feet in both camps a home.

Particularly encouraging for me was to see some of the principles of the programmable web filtering into sites such as the Guardian and the BBC which aren’t part of the Web 2.0 vowel-deprived clique. It gives me hope for the public information sector, in which I happily find myself.

So many many thanks to Edd and to everyone who supplied me with alcohol, deprived me of sleep, and talked to me about tech. I can’t wait until next year.

Women at XTech 2008

Jeni — Wed, 30 Apr 2008 21:20:46 +0100

It’s XTech 2008 next week. I’ll be there to talk about the work we at TSO have been doing with OPSI to add semantic information to the London Gazette using RDFa. It’s really interesting and timely work on all sorts of levels; you can read the abstract of the talk to get a taster and of course it’ll be published afterwards.

Anyway, I was just browsing through the schedule and it struck me how few women they were speaking. Looking at the speaker list, out of the 64 speakers, just three are women. Three! That’s not even 5%!

Looking back at last year, it was a little better, at nine out of 94, which is getting towards 10%. It wasn’t much better at XML 2007, where nine of the 82 speakers (11%) were female. At Extreme 2007, eight of the 60 speakers (13%) were women.

I wonder whether there are a low proportion of women attending these conferences generally, or whether women attend in higher proportions but don’t submit papers, or whether they submit papers but a smaller proportion are accepted.

Anyway, if you’re a woman who’s going to XTech 2008 and you want to get together to talk about kittens, drop me a line.

xtech2008

The Distributed Web

XTech 2008

Women at XTech 2008