rest

Opaque URIs != Unreadable URIs

Jeni — Sat, 25 Jul 2009 20:41:34 +0000

I’ve been talking about URIs a lot recently. One of the things that has bothered me about some of the conversations is the conflation of the concepts of “opaque URIs” and “non-human-readable URIs”. This is my argument for keeping the concepts separate.

The opacity of URIs is an important axiom in web architecture. It states that web applications must not try to pick apart URIs in order to work out information from them. Applications must not, for example, use the fact that a URI has .html at the end to infer that it resolves to an HTML document. It’s closely related to hypertext as engine of application state, in that opaque URIs should not be generated by web applications either: they must be discovered through links and the submission of forms.

But this has nothing to do with readability or hackability, both of which are extremely important for human users. Readable URIs help human users understand something about the resource that the URI is pointing to. Hackable URIs (by which I mean ones that people might manipulate by altering or removing portions of the path or query) enable human users to locate other resources that they might be interested in.

Before I go further, a couple of caveats:

I am not saying that every URI must contain a natural language identifier. An example is the URI for a school, which could include:

the name of the school
the unique reference number for the school
the record number for the school in the database that is being published on the web

Using the name of the school, as I’ve discussed, is probably a bad idea because of its lack of longevity. Using the record number for the school within the particular database that’s being published is entirely non-human-readable because there is simply no way of finding out what that would be for a given school. The unique reference number for the school, on the other hand, may be an obscure series of digits, but it is a meaningful one which renders the URI readable and hackable.

There are also times when uniquely identifying a resource using natural identifiers within the URI leads to incredibly long and complex URIs, in which case the ‘human readable’ version isn’t actually human readable. Introducing non-human-readable components is then the only option.

Back to my argument:

Why should URIs support humans doing things that applications must not? Because humans are intelligent. When humans hack a URI, they are aware that they are making a guess, taking a chance and might or might not end up at something useful. If they get a 404, or even more importantly if they get to information about something that they weren’t expecting, they are intelligent enough to recognise that the chance they took didn’t pay off. Applications aren’t intelligent. They can’t tell the difference between a right guess and a wrong guess, so it’s best not to let them guess at all.

Let me give an example. Let’s say that I’m creating a URI for a particular house. Here are two possible URIs:

http://id.example.org/house/NG9_3HZ/4
http://id.example.org/house/0aef0218

The first is readable and hackable. A human could change the house number or the postcode. They could remove the house number and expect a list of houses within the postcode. The second is not readable or hackable: there is no way to know what you would get if you changed the identifier within the URI.

Now it is true that an application accessing a site that used the URIs like the first could create those URIs programmatically whereas it couldn’t (perhaps) create a URI like the second. But if it did create the URIs programmatically it would be the fault of the application, not the fault of the URI.

As publishers, it is our responsibility to provide humans URIs that are meaningful and hackable, and to provide applications with the means of creating or identifying these URIs through forms and links. But it is not our responsibility to prevent applications from doing things that they should not do by deliberately obfuscating our URIs.

My Perfect XML-Based Publishing Platform

Jeni — Fri, 29 May 2009 21:40:20 +0000

For the last several months, I’ve been working on a project at TSO for publishing UK legislation using a native XML database (eg eXist or MarkLogic Server) with some middleware (eg Orbeon or Cocoon). It’s a powerful and flexible approach that’s built on declarative languages like XQuery, XSLT, and XML pipelines; you can see it in action with the Command and House Papers demo.

But the killer platform isn’t quite here yet, partly because the specs aren’t quite done. Both Orbeon and Cocoon use XML pipelines, but they use different languages to define them; XProc is just around the corner. XML databases are all over the place in their conformance to XQuery, its optional features and the not-quite-finalised specs for free-text searching and updating.

People talk about how productive you can be using Ruby on Rails or Django, and they work great for publishing data you can store in a relational database. What we need is a similarly easy-to-use platform for document-oriented, XML-based content. This is my wish-list.

The killer platform would have a configuration mechanism for mapping HTTP requests that it receives onto XProc pipelines. The pipeline that would be used could be based on one or more of:

the HTTP method
the requested URI
any HTTP header

The pipelines would have a signature like:

a primary ‘source’ input that encodes the HTTP method, headers and body of the request; this would use the <c:request> element used within the p:http-request XProc step
a parameter input populated by parsing the URI against a simplified version of URI templates
a primary ‘result’ output that is an XML version of the response body
a ‘response’ output that encodes the HTTP status code and headers of the response; this would use the <c:response> element used within the p:http-request XProc step
a ‘serialize’ output that holds a <c:parameters> element containing parameters for serializing the result body; possible serialisations would include serialising XSL-FO as PDF and SVG as JPEG, for example.

The pipeline engine would of course include efficient implementations of all the required steps, most importantly XSLT 2.0.

The platform would have an easy mechanism for invoking queries on its XML store through an implementation-defined step that was similar to the p:xquery XProc step. The step might have the signature:

a primary ‘query’ input for the query itself (like the ‘query’ input for p:xquery)
a parameter input for specifying the values of external variables within the query
a ‘database’ option for specifying the database to query
a primary ‘result’ output for the result of the query, this being a sequence of documents resulting from the query

The XML store itself would support:

XQuery 1.0, with no extensions to the syntax except those permitted by that specification
XQuery and XPath Full Text 1.0
XQuery Update Facility 1.0

It would also support setting up indexes on any expression for a particular kind of context node (usually an element); these would work like keys in XSLT, except that the XQuery engine would automatically detect when the index could be applied. For example, it would be possible to set up a key on a document for the expression substring(//dc:identifier, 7, 2) and if the query used exactly this expression, the index would be used.

The platform would provide an extensible architecture such that it would be possible to set up replicated XML store(s) on separate servers from the main pipeline engine. It would cache the results of queries against the XML store. It would serve up static content such as images and scripts bypassing the pipeline. It would be configured using files, so that it was easy to transfer a configuration between development and production platforms and to version control configurations through normal means.

Have you used (or developed!) anything that comes close? What’s on your wish-list?

Your Website is Your API: Quick Wins for Government Data

Jeni — Sun, 01 Feb 2009 09:28:57 +0000

This is the talk I prepared for the UKGovWeb Barcamp, in blog form. It’s probably better this way. Most of what’s written here seems blindingly obvious to me, and probably to most readers of this blog, but maybe Google will direct someone here who finds it useful.

Working with public-sector information on the web, one of the things that I take an interest in is making government data freely available for anyone to re-present, mash-up, analyse and generally do whatever they want to do. This post is born out of a feeling that the people who control data don’t realise that the smallest changes can be beneficial: they don’t need to do everything right now, just something.

There are three fundamental things that you need to do:

identify the data that you control
represent that data in a way that people can use
expose the data to the wider world

but you can choose the degree to which you do each of these things.

Identify

Take a look at what data you have some kind of responsibility for or control over. You might be a PDF containing a table of schools in the local area and their intakes over the last couple of years. You might have a spreadsheet of the amount of money assigned to maintaining the playgrounds within the borough. You might have a database of company information. You might have a set of HTML agendas for court cases.

The first step is simply to identify what the information is about. Schools, playgrounds, companies, court cases — each row in your table or spreadsheet or database, or each section in your document will be about something. We call this a resource.

To play nicely with the web, every resource should have an identifier. A Uniform Resource Identifier. A URI. That URI tells us where we can find information about the resource (we’ll get to what those look like later). So your second step is to work out URIs for each of your resources.

Now, there are actually three levels of URIs that you can care about:

identifier URIs
document URIs
representation URIs

You probably already have document and representation URIs on your web server. Representation URIs are URIs for particular formats and languages and views of the information that you make available. Document URIs are typically the same URI without an extension; web servers use content negotiation to work out which representation to serve up when a web browser asks for the page at a particular document URI.

So you already have a URI for the PDF that contains the table of schools, for the Excel spreadsheet about the playgrounds. You already have URIs for the results of a particular query on your database, and of course the HTML pages that you deliver have URIs already. That’s all in place. You don’t want to change it.

But identifier URIs are what are really important when it comes to opening up your data. They shift the focus from the documents that you serve to the resources that they are about. By assigning URIs to resources, you enable other people to talk about them. Even if that’s all you do, you have done good.

For example, if Companies House stated that companies could be referred to using URIs of the form http://www.companieshouse.co.uk/id/company/{registeredNumber} then other people who needed to talk about companies (websites containing customer feedback, monitoring companies going into receivership, displaying stock price information, whatever) could use these URIs whenever they referred to a company. If all websites that make data available about companies point to the same identifier for a company, then it’s possible to pull that data together very easily.

Now the URIs that you use should be short, clean, readable, hackable, hierarchical and so on. If you can, you should use a natural identifier for the resource within the URI for that resource. So URIs for registered companies should use their registered number. URIs for schools should use the school’s unique reference number (URN). URIs for playgrounds could use the name of the playground (scoped within the council responsible for the playground). URIs for court cases should include the court, the year, and the case number. And so on.

Remember as you’re creating these identifier URIs that they are nothing to do with the structure of your website or the user’s experience of navigating through your website. For navigation, you might want to group schools into primary, secondary and sixth-form, but you shouldn’t do that in the identifier URIs. To help decide, imagine someone wanting to construct a URI and the information that they need to do so. If any of the information they need can be derived from other information (as a school’s type can be derived from its URN), leave it out.

When you’re doing this, you might realise that actually you shouldn’t be the one in control of these URIs. If you’re not the one assigning the registered number, URN or case number then there’s probably a higher authority that does assign those (real-world) identifiers. Don’t let that stop you creating URIs — you’ll still find them useful for identifying your information about that particular resource — but do look to see if there are existing URIs that you could point to and reuse whatever scheme they’re using if there are.

Represent

So I said in the last section that assigning URIs to resources was useful. And it is. But it’s even more useful if you provide some kind of response when someone requests those URIs. A request for a URI can be done by a web browser or one of those search-engine-spider-things that crawls the web looking for data. Requests are done on the web using HTTP (hypertext transfer protocol), specifically using a GET request, which means “get this resource”.

When a web server receives a request, it sends back a response. The first part of the response is a status code that tells the browser, spider, or whatever issued the request, generally what kind of response it is. Now when a browser says “get this company” or “get this school” a web server should either respond with a 404 Not Found response or a 303 See Other response.

If the company or school doesn’t exist, a web server should respond with a 404 Not Found response. It’s actually really useful to give appropriate 404 Not Found responses, because it tells whoever made the request that the resource (company/school/playground/court case) doesn’t exist. This can act as simple validation: if I’m building a site that parents can use to rate schools, and a parent enters a URN into a form, I can construct a URI based on that URN, try to GET the information about that school, and if I get a 404 Not Found response then I know that the parent has entered an invalid URN.

If the company or school exists, a web server should respond with a 303 See Other response that points the browser to a document URI that contains information about the company or school. After all, the web server can’t very well deliver the company or school itself into your lap; all it can do is give you information about it. 303 See Other means “if you want information about that, see that other thing over there instead”. The “other thing over there” will be a document of some kind. It might be the PDF that contains information about the school, or the spreadsheet that contains information about the playground.

Simply giving a yes-this-exists or no-this-doesn’t-exist response is useful. Even if that’s all you do, you have done good.

It’s even more useful, though, if you can make the information that you have about the school, playground, company, court case or whatever, available in a format that can be processed by a computer reasonably easily. PDFs are really really hard to extract information from, so do everything you can not to use PDFs. Word documents and Excel spreadsheets are next worse; if you have to use them, keep them really really simple and definitely don’t use Word Art or embed images to display your data.

You should always make your data available in HTML. Try to make it as clean and regular as you can; use microformats to indicate information about people, places and events. If you want to push the boat out, use RDFa to mark up the data in your page even more explicitly.

The great thing about HTML is that it’s human readable as well as (if you do it well) machine readable. You can also make your data available in explicitly machine-readable forms as well if you want: XML, JSON, RDF/XML, whatever floats your boat. If there are already standard formats or ontologies for the kind of data that you’re making available, then use them, certainly, but it’s very likely that there aren’t. And in comparison to the nightmare of extracting anything useful from a PDF, it’s easy to transform between different formats, so you only have to concern yourself with different formats if you want to.

If you do provide multiple formats for your data, you should use server-driven content negotiation to deliver the data in an appropriate format to whatever’s requesting it. So a web browser will request HTML; a semantic web crawler will request RDF/XML; a Javascript program will request JSON and so on. The 200 OK response that the web server sends with your data should include a Content-Location header that gives the representation URI of whichever format is being returned, and a Vary header that tells caches how it’s decided which representation to serve up.

Expose

All the good work identifying resources and representing them comes to naught if you don’t expose it. You can (and should!) tell other people about the URIs that you’ve developed, but the best way to give them exposure is to use them yourself, within your website. Simply using the URIs within your website gives them exposure. Even if that’s all you do, you have done good. People who are interested in linking to you will look at your site and they will learn about your URI scheme from your use of it.

The identifier URIs that you’ve created might not be particularly easy to generate. For example, with the URI scheme that I suggested above for Companies House, unless you happen to know that Tesco Plc’s registered company number is 00445790, you’re not going to be able to get to information about them. So you should have a way of searching based on something that people will know, such as the name of the company. Use an HTML search form that makes GET requests like

http://www.companieshouse.gov.uk/company?name=Tesco Plc

The response should be a 302 Found that redirects (using the Location header) to the true identifier URI for the company (http://www.companieshouse.gov.uk/id/company/00445790). If it’s not possible to identify a single resource from the search string (for example, there are lots of companies with ‘Tesco’ in their name), then the correct response is a 300 Multiple Choices that provides a list of links to the possible URIs (in HTML).

There are other ways to help people find your data. If there aren’t gazillions of resources, you can list the URIs within your sitemap, which will make them discoverable by search engines. You can also list them on web pages and, especially for data that’s constantly updating, in (Atom) feeds which you link to from your HTML pages. Use metadata within the pages and feeds to help the consumers of your data work out what’s relevant to them.

To help even more, slice your Atom feeds into portions that different consumers of your data are going to be interested in. Slice by type, by area, by subject. That way people can stay up to date with just the resources that they’re interested in, and not be bothered with information about those that are irrelevant to them.

That’s It

What I’ve tried to describe here is the minimum that you need to do to help people use the information you have, and some of the other things that you can do to make it even more useful. Here are some things that you shouldn’t do:

don’t wait for someone else to define a URI scheme for the things that you want to talk about
don’t wait for someone else to define an XML schema or RDF ontology for your data
don’t wait until you can find the time and money to do it all “properly”

Just do what you can, now.

URL design for searches and queries

Jeni — Sun, 12 Aug 2007 22:00:00 +0000

Another fascinating post from Bill de hÓra, this time on URL design for resources:

Let’s take editing some resource, like a document, and let’s look at browsers and HTML forms in particular, which don’t a do a good job of allowing you to cleanly affect resource state. What you would like to do in this suboptimal environment is provide an “edit-uri” of some kind. There are basically 5 options for this; here they are going from most to least desirable

Uniform method. Alter the state by sending a PUT to the document’s URL. The edit-uri is the resource URL. URL format: http://example.org/document/xyz

Function passing. Allow the document resource to accept a function as an argument. URL format: http://example.org/document/xyz?f=edit

Surrogate. Create another resource that will accept edits on behalf of the document. URL format: http://example.org/document/xyz/edit

CGI/RPC explicit: send a POST to an “edit-document” script passing the id of the document as a argument. URL format: http://example.org/edit-document?id=xyz

CGI/RPC stateful: send a POST to an “edit-document” script and fetch the id of the document from server state, or a cookie. URL format: http://example.org/edit-document

My current task at work is to look at how to add RDFa to a website that is completely driven by “CGI/RPC explicit” URLs. That includes URLs for the resources themselves, by the way, we’re not even talking about edit URLs here. Take a look at the URL for this page, for example (this isn’t the actual website that I’m working on, but it’s more or less the same in terms of URL design). The URL is

http://www.statutelaw.gov.uk/content.aspx?LegType=All+Legislation&title=wine&Year=2007&searchEnacted=0&extentMatchOnly=0&confersPower=0&blanketAmendment=0&sortAlpha=0&TYPE=QS&PageNumber=1&NavFrom=0&parentActiveTextDocId=3032571&ActiveTextDocId=3032571&filesize=16218

So here I am trying to construct RDF examples, and all the URLs look like this mess. What URI am I supposed to use in RDF to talk about the resource itself, rather than a particular view (table of contents, actual content, etc) of that resource?

In this case, the thing that identifies the resource in the URL is the value of the ActiveTextDocId request parameter: you can do

http://www.statutelaw.gov.uk/content.aspx?ActiveTextDocId=3032571

and see the same legislation; this could be mapped to a resource-oriented URL such as

http://www.statutelaw.gov.uk/legislation/3032571

very easily. It isn’t, but it could.

However, doing that, you do lose some context about what the original search was that led you to this page. In the case of the above URI, the fact that I searched for all 2007 legislation with “wine” in the title gets lost. And this is important because the breadcrumb on the page has to take me back to that original search.

Now, you could argue that this is bad website design: after all, you can navigate back to a search page using *gasp* the Back button, and not doing so just adds unnecessary items to your history. But what about providing previous and next links for navigating through the items found in a search? There, surely, you do need some state information that indicates how we got to this particular item?

Well, no. When you’re navigating through the results of a search, the primary resource that you’re viewing is the collection of items that have been identified by the search. Even if you’re just viewing one of the items in that collection, if the collection still matters then that item should be viewed as just a subresource of the collection.

In this case, the search has three fields — title, year and (legislation) number — so the search URL has three parts after the initial one. The general scheme (using URI template syntax) is

http://www.statutelaw.gov.uk/search/{title}/{year}/{number}

So here, I could use

http://www.statutelaw.gov.uk/search/wine/2007/_

as a URL that would give me a list of all the 2007 legislation with any number whose title contained “wine”, and then

http://www.statutelaw.gov.uk/search/wine/2007/_/1

would show me the first piece of legislation found in the context of that search, with a next button taking me to

http://www.statutelaw.gov.uk/search/wine/2007/_/2

An individual resource itself could occur in many searches, and thus you would get many URLs for the resource in the context of those particular searches, but that’s OK so long as there is one URI that identifies the resource itself. You should link to the actual resource from the page, of course, both in a <link> in the header and a <a> in the body (it feels like there should be a “this is the real resource” value for the rel attribute of <link> or <a>, but I don’t know of one).

The kind of URL above works fine when you have a fixed number of fields for searches, but what if you’re doing more complicated searches: something that requires a proper query language? Well, you can stuff a query language into a URL. See Dare Obasanjo’s comparison of Google and Astoria APIs for queries to see what that looks like.

Alternatively, several years ago Paul Prescod introduced me to the notion that the query itself is a resource — it’s something that you’ll probably want to save and edit — and can be assigned a unique identifier in the same way as other resources. So you visit

http://www.example.com/queries/new

to create a new query, which gets assigned the URL

http://www.example.com/queries/4328

and you can then visit that page to see a list of the results of the query. Unlike with a simple search, the query parameters themselves don’t get used in the URL: they’re stored on the server. So you can’t hack the URL to change the query, but you do have a simple URL that you can easily share with other people if you want to.

View-Model-Template

Jeni — Fri, 27 Jul 2007 10:02:04 +0000

I don’t know anything about Struts 1, but Bill de hÓra’s recent post has got some interesting web-application-design tips. There were two particular bits that spoke to me:

struts-config.xml struts-config tries to capture primarily the flow of application state on the server, by being an awkward representation of a call graph. In doing it misses a key aspect of the web - hypertext. In web architecture, HTML hypertext on the client is the engine of application state, not an XML file on the server.

In other words (I think) in web applications your state in the page you’re on and taking action is about following the links (or submitting the forms) on the page. Your actions (and therefore the transitions between different states) are determined by what links and forms are on the page. But in fact, URLs should be hackable, and transitions unlimited. When you design the application what you really need to think about are the tasks the users want to achieve (and therefore the transitions that they might want to make) rather than the possible state transitions.

On the web, a suitable pattern is View, Model, Template [rather than Model, View, Controller (MVC)]. A request to a URL is dispatched to a View. This View calls into the Model, performs manipulations and prepares data for output. The data is passed to a Template that is rendered an [sic] emitted as a response. ideally [sic] in web frameworks, the controller is hidden from view. Note that this framework style is often called MVC anyway, confusing matters somewhat; The key differences are that Views and Templates are cohesive and Controllers are pushed down into the framework infrastructure.

I’ve been thinking recently about whether and how XSLT might fit into a Ruby on Rails set-up. In RoR, the controller usually either queries the database (via the model) to set up instance variables, and then renders a (template) view, or updates the database (via the model) and redirects to another view. The templates (for (X)HTML) use fairly standard <% ... %> placeholders to hold code and insert values.

I’ve spent most of my professional life cursing (X)HTML documents with <% ... %> in, because they use unescaped less-than-signs and therefore can’t be generated or processed by XML tools, particularly XSLT. There’s an advantage of having templates that are themselves well-formed, not least that you can easily process the templates themselves (for example to generate, update or document them). Plus if your templates are declarative, rather than containing embedded code, you aren’t tied to a particular framework: I could move templates from Ruby on Rails to Django and they wouldn’t need modification. When I think “declarative templates”, I think “XSLT”.

The other advantage of using XSLT is that it can be used on the client side as well as the server side. So there’s the possibility of moving that rendering from one server to client completely or using it on particular clients, perhaps in an AJAX set-up, while having the same stylesheets on the server for those browsers that don’t support client-side XSLT.

You still need a way of getting the data from the model into the stylesheet, which can be done through a combination of XML and parameters. The XML is itself a view of the model, of course, but if you’ve got any kind of intention to make your web application mashable, you’re going to want to generate XML, probably Atom, anyway (yeah, or JSON, but it’s easy enough to get from XML to JSON using XSLT too). If you add caching to the equation, this approach might help reduce database requests.

So I think that using XSLT as a templating language, even within a RoR framework, has at least something going for it. What I hope is that I’m not falling into the “when you’ve got a hammer everything looks like a nail” trap.

XTech 2007: Thursday 17th May Afternoon

Jeni — Sun, 27 May 2007 22:03:24 +0000

UPDATE: Dare Obasanjo has written an interesting critique on using the Atom Publishing Protocol as the basis for general purpose sharing of data in the way that the Google Data API does.

Thursday afternoon had a few really interesting talks. I learned about the Google Data API (no longer called gData); Oracle’s use of XLink to represent relationships between documents, and the requirements that entails; using XSLT to create JSON to use Exhibit widgets; and using XMPP to enhance instant messaging.

Google Data API

Frank Mantek

The Google Data API is the unified API that Google offers to all its services, such as Google Base, Blogger, Google Calendar, Google Spreadsheets and so on.

Frank talked about how awful SOAP/WSDL is, in particular how two services developed in different platforms can’t talk to each other (which one might imagine is rather the point of Web Services). (Later, when challenged by a Microsoft guy about this claim, he revealed that he’d been a major developer of the SOAP/WSDL stuff at Microsoft, so knew exactly what he was talking about from bitter experience.)

So the Google Data API is a RESTful API, using the Atom Publishing Protocol with a few additions:

extra data model
querying
concurrency control
extra authentication

What this basically means is that you can query any of the Google services using HTTP, and get back an Atom document. The URI can contain queries (the precise nature of which depend on the service; Google Base, for example, uses a single URI request parameter that has a complex internal query syntax), and you get back the feed with the items that you’d requested. The Atom items themselves have the basic Atom elements, but then a bunch of service-specific elements that provide the extra information you need.

Listening to this talk I finally got what Tim Bray was talking about at the XML Summer School a couple of years ago: REST gives us verbs and Atom gives us objects and lists of objects. I didn’t get it before, because, after all, aren’t all XML documents objects? But I think the point is that Atom has a lot of the mechanics that you need for talking about objects built into it, and the extensibility necessary for adding your own information to it (which is what each of Google’s services are doing).

The really interesting part of the talk was where Frank started talking about what the problems (still) are. The problems I noted were:

Atom’s verbose
Google have to use <category> to indicate the kind of thing they’re representing (as opposed to using the document element which is what you’d do with normal XML documents)
the rel attribute is too vague
they made up their own markup languages, rather than reusing existing standards
they should be using ETags for concurrency control
they haven’t got any versioning (eek)
incremental updates are a problem; they don’t want to serve the whole Atom feed (to a mobile device) when only a small amount has changed, so what they do is have several feeds, each of which reveals a different part of the information

From Trees to Graphs: Evolving XML for building enterprise applications

Ravi Murthy

Ravi Murthy talked about the provision for defining links between documents in Oracle’s database, and their consequent requirements. Information Oracle’s XML database has a file system abstraction (every XML ‘object’ has a file path) with access control, versioning, metadata and protocol access. Within an XML ‘object’ stored in the database, they use XLink to represent the relationships with other objects. When you export the XML, the XLinks get resolved to create the XML document.

Using XLink to represent relationships between documents brings a whole new set of constraints that you might want to express in a schema language, or annotations that you can use to describe the links (depending on how you look at it):

The type of the linked resource (eg the document element’s name, substitution group or XSD type)
The scope of a particular reference, similar to the scoping of XSD’s identity constraints
That a particular link is acyclic (eg, given an XPath expression, keep evaluating it and make sure you never get back to where you started)
The kind of a link, one of:
- hard: the target of the link must exist, and cannot be deleted while this resource exists (but can be renamed) — these are similar to links in normal databases
- symbolic: trust the file path specified by the link and only resolve it on demand
- weak: like a hard link, except the target can be deleted, in which case the link becomes symbolic
The versioning of a link, whether it points to the “current” version of a resource or a specific version

These extra constraints are expressed as annotations on the definitions of xlink:href attributes in XSD schemas for the documents held in the database.

Ravi also talked a bit about expressing decomposition rules: how an XML document should be shredded when it gets put into the database. They use XPath to specify rules that indicate that particular elements should be placed at a particular filepath.

I was really flattered in the tea break. Chatting with a guy called Phil working at the University of Bath, who politely asked about my presentation, and after I’d explained how it was all to do with overlapping markup and that kind of hard-core theory he said: “You don’t look like a markup geek”. Me: “What, because I’m a girl?”. Him: “No, no, that’s not what I meant. You just look more Web 2.0-ey.” Max was there at the time, and labelled me “the Geekess of XSLT”, which I think clarified things. (Actually most of the people at XTech this year were Web 2.0-ey rather than markup geeks, but I’m glad I looked as though I fitted in.)

XML-powered Exhibit: A Case Study of JSON & XML Coexistence

Chimezie Ogbuji

“What’s Exhibit?” I hear you ask. Or maybe you’re more with-it than I am, but that’s what I was asking. Chimezie never really explained, but I kinda gathered that it’s a funky AJAX toolset for creating views of data by importing scripts and using magical IDs and extension attributes within web pages. The other phrase that Chimezie dropped in was Rich Web Application Backplane, which again I hadn’t heard of. Even having read the W3C Note, I still don’t get it. Ho hum.

Anyway, Chimezie made the point that while entering data using XForms is great, it’s too heavy-weight for viewing that data. Exhibit gives a lot more flexibility (take a look at the US presidents example), which enables users to explore data more freely. In Exhibit pages, you provide a JSON schema for your data, a number of lenses/views/widgets that you can use to view the data, then you embed the widgets in the HTML page and point it at the data source. The JSON schema indicates the type of a particular property (eg “country”), and gives labels for it (including a plural label (“countries”) and a reverse label (“country of”)) that it uses in the widgets.

But that requires JSON, right? Chimezie showed how easy it is (and it’s really easy) to transform data-oriented XML into JSON using XSLT.

You know, there are all these cool ways out there for viewing information, I just wish I had some really meaty data to use them on! Timelines are one thing, but I’d also love to find some data to employ in Gapminder or even in an interface like the one for the music of Philip Glass. Perhaps I should just mine Google Base, but I’d like it to be something personally or collectively useful.

Real-time user-to-user web with Mozilla and XMPP

Massimiliano Mirra

This talk was strong on motivation — the requirement to enhance basic instant messaging functionality — and strong on demonstration, with Massimiliano chatting and playing with a pre-programmed bot, but really weak on the technical details. It was only through the post-talk questions that we learned that what we’d seen was based on XMPP (the Extensible Messaging and Presence Protocol), which allowed DOM events to be passed between clients. Have to read the paper if you want to learn more.