uri

Creating Linked Data - Part II: Defining URIs

Jeni — Sun, 22 Nov 2009 17:23:34 +0000

This is the second instalment in a series of posts about how to create linked data from existing data sets, using traffic count data as an example. In the last instalment, I talked about analysing and modelling data. This instalment discusses the creation of URIs for the various things that have been identified within the model.

This part of the process is the same as what you’d do if you were simply creating a RESTful API to a website. The principal is that everything has a URI, and if you resolve that URI you get information about the thing.

For the data.gov.uk site, we now have some guidelines about the design of URIs for the UK public sector. Basically, URIs for things should look like:

http://{sector}.data.gov.uk/id/{type of thing}/{thing identifier}

There’ll be plenty of examples in what follows.

Areas

Some of the things that we’ve identified as being part of the traffic count dataset already have centrally-defined identifiers. As part of other data.gov.uk work, we’ve defined URIs for administrative areas like countries, regions, local authority districts and local authorities. The templates for these URIs are:

http://statistics.data.gov.uk/id/country/{ONS code}
http://statistics.data.gov.uk/id/government-office-region/{ONS code}
http://statistics.data.gov.uk/id/local-authority-district/{ONS code}
http://statistics.data.gov.uk/id/local-authority/{ONS code}

We can use these identifiers directly for the regions, districts and local authorities. But there’s a problem with the country URI: we don’t have the ONS code for the country, only the name of the country. Fortunately, we’ve also defined URIs with this pattern:

http://statistics.data.gov.uk/id/country?name={country name}
http://statistics.data.gov.uk/id/government-office-region?name={region name}
http://statistics.data.gov.uk/id/local-authority-district?name={district name}
http://statistics.data.gov.uk/id/local-authority?name={authority name}

so in this situation we can use the name-based country URI and we’ll get redirected to the canonical, code-based URI.

Local authorities actually have two codes within the dataset that we have: the ONS code and a DfT code. I can well imagine that other datasets from the Department for Transport will only reference the DfT code, so it’s a good idea to create URIs that are based on these codes; later on, we can state that the two identifiers actually mean exactly the same thing.

http://transport.data.gov.uk/id/local-authority-district/{DfT code}
http://transport.data.gov.uk/id/local-authority/{DfT code}

So given the record:

"England","North West","B",4315.00,"00BZ","St.Helens Metropolitan Borough Council",
4,"U",,"Unclassified Urban",,
,352100,398200,
7/6/2001 00:00:00,"N",7,1,0,5,1,0,0,0,0,0,0,0,0,6

the URIs we’ve defined so far are:

http://statistics.data.gov.uk/id/country?name=England
http://statistics.data.gov.uk/id/government-office-region/B
http://statistics.data.gov.uk/id/local-authority-district/00BZ
http://statistics.data.gov.uk/id/local-authority/00BZ
http://transport.data.gov.uk/id/local-authority-district/4315
http://transport.data.gov.uk/id/local-authority/4315

Roads

Now we’re onto things that aren’t defined already. First is roads. If there’s a road number, the obvious thing to use is that road number; something like:

http://transport.data.gov.uk/id/road/{road number}

For example:

http://transport.data.gov.uk/id/road/B3178

If there isn’t a road number, we’ll have to construct a URI. Since each count point is on one particular road, we can use the identifier of the count point to identify the road, so:

http://transport.data.gov.uk/id/road/{class}-{count point number}

For example:

http://transport.data.gov.uk/id/road/U-4

Count Points

Count points can be identified through their number, so it makes sense to use that in the URI:

http://transport.data.gov.uk/id/traffic-count-point/{count point number}

For example:

http://transport.data.gov.uk/id/traffic-count-point/4

Counts

The counts themselves don’t have their own identifiers, but they can be identified through a combination of the count point that they’re associated with, the direction of travel of the traffic that’s being counted, and the date and time that the count is made. So we can create a URI that combines these things. To aid hackability, I’m going to build on top of the traffic count point URI that we’ve already defined:

http://transport.data.gov.uk/id/traffic-count-point/{count point number}/direction/{direction}/hour/{time}

For example:

http://transport.data.gov.uk/id/traffic-count-point/4/direction/N/hour/2001-06-07T07:00:00

Observations

Again, observations build on top of the counts by adding a vehicle type to the mix, so we can construct URIs that reflect that:

http://transport.data.gov.uk/id/traffic-count-point/{count point number}/direction/{direction}/hour/{time}/type/{vehicle type}

For example:

http://transport.data.gov.uk/id/traffic-count-point/4/direction/N/hour/2001-06-07T07:00:00/type/motor-vehicle

Road Categories

Road categories are a bit different from the kinds of things that we’ve been talking about so far: they are concepts. For these URIs we use a slightly different pattern from the URIs above: /def/ rather than /id/. For road categories we can use:

http://transport.data.gov.uk/def/road-category/{category}

For example:

http://transport.data.gov.uk/def/road-category/motorway

Vehicle Types

Vehicle types are also concepts, so have similar URIs:

http://transport.data.gov.uk/def/vehicle-category/{type}

For example:

http://transport.data.gov.uk/def/vehicle-category/HGVa5

Cardinal Directions

Cardinal directions are also concepts, but really they are global concepts, not specific to transport, or even to the UK. So it feels a bit strange to use URIs for them that imply that they somehow belong to data.gov.uk.

Fortunately, for this kind of general concept we can use URIs defined by DBPedia. DBPedia is a linked data view on Wikipedia, so it has URIs for everything that Wikipedia has a page about, making it an excellent general purpose resource. The relevant URIs for the cardinal directions are:

http://dbpedia.org/resource/North
http://dbpedia.org/resource/South
http://dbpedia.org/resource/East
http://dbpedia.org/resource/West

so that’s what we’ll use.

Dates, Times and Periods

For dates, times and periods, we can use the URIs provided by another general-purpose linked data resource: placetime.com. URIs for instants have the pattern:

http://placetime.com/instant/gregorian/{dateTime}

while periods have the pattern:

http://placetime.com/interval/gregorian/{dateTime}/{duration}

So the hour from 7-8am on 7th June 2001 would be:

http://placetime.com/interval/gregorian/2001-06-07T07:00:00/PT1H

and the year 2001 would be:

http://placetime.com/interval/gregorian/2001-01-01T00:00:00/P1Y

The thing is that the latter isn’t particularly approachable. Calendar years are used all over the place, so it would be nice to have a set of URIs for them that we use consistently. Again, DBPedia provides URIs for every year, such as:

http://dbpedia.org/resource/2001

so where we need to refer to a calendar year, it would be good to reuse that.

And that completes the sets of URIs that we need for this data. Stay tuned.

The Real Deal: data.gov.uk

Jeni — Sun, 26 Jul 2009 16:38:54 +0100

I’m sure that you’ve noticed that my recent posts have been somewhat obsessed with publishing and using public sector information. It’s because I’ve somehow been sucked into the work going on within the UK government, with Tim Berners-Lee and Nigel Shadbolt advising, to publish its data as linked data.

My recent blog posts about publishing data using Talis have actually been a front for much more complex work that I’ve been doing with a different data set.

As an early demonstration of how existing government data sets might be turned into linked data, a few weeks ago I was given a CSV file containing road traffic counts; the raw data that lies behind the traffic flow information available on the Department for Transport website. The data is really interesting and ripe for visualisations and analysis. For each hour of particular days each year, at particular points on many roads within the UK, the Department for Transport measures the number of bicycles, motorbikes, cars, vans, buses and HGVs of various types that roll past in each direction. The data contains information about:

the count of each of the various classes of traffic that pass the point in a particular direction on a particular hour of a particular day
the points at which these measurements were taken
the roads on which the points are situated
the areas in which the points are situated
the local authority that is in charge of these areas
the region that the area is in
the country that the region is in

The challenge was to turn the 386Mb CSV file into linked data. The result is up and available for you to look at; a good starting point is http://geo.data.gov.uk/0/country. Just follow the links from there.

With a few false starts and mis-steps, this is the process that I went through:

Tidied the CSV file so that it could be processed using awk. That meant replacing the commas that were delimiters with |s. It also meant removing a couple of weird ^M characters that had snuck into the file.
Examined the data and came up with an informal ontology and prototype URI scheme.
Created a bunch of awk scripts to extract different data from the files and create RDF/XML from it.
Ran the scripts to create RDF/XML.
Uploaded the data into a Talis store.
Created appropriate PHP for the data and put it into a proxy server.

Some of this has been covered by my recent posts, so I’m just going to talk about a few of these steps in a bit more detail.

First, the URIs. Frankly, they’re an experiment to see how it plays. The templates are:

countries: http://geo.data.gov.uk/0/id/country/{name}, eg http://geo.data.gov.uk/0/id/country/england
regions: http://geo.data.gov.uk/0/id/region/{name}, eg http://geo.data.gov.uk/0/id/region/north-west
areas: http://geo.data.gov.uk/0/id/area/{ONS code}, eg http://geo.data.gov.uk/0/id/area/00KA
local authorities: http://local-government.data.gov.uk/0/id/local-authority/{ONS code for area}, eg http://local-government.data.gov.uk/0/id/local-authority/00KA
roads: http://transport.data.gov.uk/0/id/road/{name} or http://transport.data.gov.uk/0/id/road/U-{random number}, eg http://transport.data.gov.uk/0/id/road/M5
traffic count points: http://transport.data.gov.uk/0/id/traffic-count-point/{number}, eg http://transport.data.gov.uk/0/id/traffic-count-point/36195
traffic counts: http://transport.data.gov.uk/0/id/traffic-count/{point number}/{direction}/{date}/{hour}/{traffic type}, eg http://transport.data.gov.uk/0/id/traffic-count/4/N/2008-06-05/08:00:00/HGVr2

The subdomains are one way of subdividing the vast set of public sector information into vague categories that might be handled by different departments, without using the (highly changeable) department names in the URI. The /0 portion of each URI is a version number: these URIs are experimental and liable to be unsupported in the future so they’re marked with a version 0. The /id portion of each URI indicates that these are URIs for non-information resources; the response is a 303 See Other redirect to the same URIs but without the /id.

After the /id, the URIs follow a common pattern of naming a class of resource, followed by an appropriate identifier for that resource. The identifiers themselves are designed to be unique, unlikely to change, and human readable.

The ontologies, well, actually they don’t exist as yet except in my head. It’s been more important to make the data available than to provide ontologies for it. Triplestores and SPARQL queries work without ontologies; indeed you have to go out of your way to find applications that actually reason with them. Like schemas for XML documents, they’re not absolutely essential, but useful for documentation purposes and potentially useful for applications.

There are, though, a couple of SKOS schemes for categorising roads and vehicle types. These are available via:

http://transport.data.gov.uk/0/category/road
http://transport.data.gov.uk/0/category/vehicle

They were informed by the British Roads FAQ and the data definitions from the Department for Transport. I heartily recommend a read; it’s scintillating stuff!

Anyway, with this size of file, and the kind of processing that needed to be done with it, the simple XSLT that I talked about previously for extracting data out of CSV files just wasn’t going to cut it. Awk, on the other hand, is designed for this kind of processing. Most of the RDF/XML could be generated by collecting unique values from the file. For example, to generate the RDF/XML for the regions I used:

BEGIN { 
  FS = "|";
  print "<rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"";
  print "  xmlns:rdfs=\"http://www.w3.org/2000/01/rdf-schema#\"";
  print "  xmlns:g=\"http://geo.data.gov.uk/0/ontology/geo#\">";
}
FNR > 1 {
  countries[$2] = substr($1, 2, length($1) - 2);
  regions[$2] = substr($2, 2, length($2) - 2);
  codes[$2] = substr($3, 2, length($3) - 2);
}
END { 
  for (region in regions) {
    country = countries[region];
    name = regions[region];
    code = codes[region];
    path = tolower(name);
    gsub(" ", "-", path);
    print "<g:Region rdf:about=\"http://geo.data.gov.uk/0/id/region/" path "\">";
    print "  <rdfs:label>" name "</rdfs:label>";
    print "  <g:isInCountry>";
    print "    <g:Country rdf:about=\"http://geo.data.gov.uk/0/id/country/" tolower(country) "\">";
    print "      <g:hasRegion rdf:resource=\"http://geo.data.gov.uk/0/id/region/" path "\" />";
    print "    </g:Country>";
    print "  </g:isInCountry>";
    if (code != "") {
      print "  <g:ONScode rdf:datatype=\"http://www.w3.org/2001/XMLSchema#NCName\">" code "</g:ONScode>";
    }
    print "</g:Region>";
  }
  print "</rdf:RDF>"; 
}

This generated RDF/XML that looks like:

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
  xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
  xmlns:g="http://geo.data.gov.uk/0/ontology/geo#">
<g:Region rdf:about="http://geo.data.gov.uk/0/id/region/london">
  <rdfs:label>London</rdfs:label>
  <g:isInCountry>
    <g:Country rdf:about="http://geo.data.gov.uk/0/id/country/england">
      <g:hasRegion rdf:resource="http://geo.data.gov.uk/0/id/region/london" />
    </g:Country>
  </g:isInCountry>
  <g:ONScode rdf:datatype="http://www.w3.org/2001/XMLSchema#NCName">H</g:ONScode>
</g:Region>
<g:Region rdf:about="http://geo.data.gov.uk/0/id/region/yorkshire-and-the-humber">
  <rdfs:label>Yorkshire and The Humber</rdfs:label>
  <g:isInCountry>
    <g:Country rdf:about="http://geo.data.gov.uk/0/id/country/england">
      <g:hasRegion rdf:resource="http://geo.data.gov.uk/0/id/region/yorkshire-and-the-humber" />
    </g:Country>
  </g:isInCountry>
  <g:ONScode rdf:datatype="http://www.w3.org/2001/XMLSchema#NCName">D</g:ONScode>
</g:Region>
...
</rdf:RDF>

In other cases, I needed to split up the RDF/XML that was generated into several files. Uploads to Talis of more than about 2Mb cause the upload to fail. The traffic count point RDF/XML needed to be split into 13 separate files. The traffic counts themselves… well, I haven’t managed to do it all yet but to give you an idea, the 2008 data alone generated 1800 RDF/XML files, each about 1.6Mb in size and each taking about a minute to upload. What’s there now is all the 2008 data, and the overall motor vehicle counts from all the years. More will be added gradually.

The awk script that generates the count data in separate files is:

BEGIN { 
  FS = "|";
  fileCount = 0;
  countCount = 99999;
  curlFile = "traffic-counts.curl.sh";
}
FNR > 1 && $15 ~ /\/2008 / {
  countCount += 1;
  if (countCount > 200) {
    if (fileCount != 0) {
      print "</rdf:RDF>" > fileName; 
      close(fileName);
    }
    countCount = 0;
    fileCount += 1;
    fileName = "traffic-counts/traffic-counts." fileCount ".rdf";
    print "creating", fileName;
    print "echo loading", fileName > curlFile;
    print "curl -H \"Content-type: application/rdf+xml\" -o progress.txt --digest -u username:password --data-binary @" fileName " http://api.talis.com/stores/transport/meta" > curlFile;

    print "<?xml version=\"1.0\" encoding=\"ASCII\"?>" > fileName;
    print "<rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"" > fileName;
    print "  xmlns:rdfs=\"http://www.w3.org/2000/01/rdf-schema#\"" > fileName;
    print "  xmlns:xsd=\"http://www.w3.org/2001/XMLSchema#\"" > fileName;
    print "  xmlns:t=\"http://transport.data.gov.uk/0/ontology/traffic#\"" > fileName;
    print "  xml:base=\"http://transport.data.gov.uk/0/id/traffic-count/\">" > fileName;
  }

  cp = $7;
  date = $15;
  direction = substr($16, 2, length($16) - 2);
  split(date, dateFields, " ");
  date = dateFields[1];
  split(date, dateFields, "/");
  date = sprintf("%04d-%02d-%02d", dateFields[3], dateFields[2], dateFields[1]);
  hour = sprintf("%02d:00:00", $17);
  base = "http://transport.data.gov.uk/0/id/traffic-count/" cp "/" direction "/" date "/" hour;

  cycles = $18;
  motorbikes = $19;
  ...

  print "<t:Count rdf:about=\"" base "/cycle\">" > fileName;
  print "  <t:point>" > fileName;
  print "    <t:CountPoint rdf:about=\"http://transport.data.gov.uk/0/id/traffic-count-point/" cp "\">" > fileName;
  print "      <t:count rdf:resource=\"" base "/cycle\" />" > fileName;
  print "    </t:CountPoint>" > fileName;
  print "  </t:point>" > fileName;
  print "  <t:hour rdf:datatype=\"http://www.w3.org/2001/XMLSchema#dateTime\">" date "T" hour "</t:hour>" > fileName;
  print "  <t:direction>" direction "</t:direction>" > fileName;
  print "  <t:category rdf:resource=\"http://transport.data.gov.uk/0/category/bicycle\" />" > fileName;
  print "  <rdf:value  rdf:datatype=\"http://www.w3.org/2001/XMLSchema#integer\">" cycles "</rdf:value>" > fileName;
  print "</t:Count>" > fileName;
  print "<t:Count rdf:about=\"" base "/motorbike\">" > fileName;
  print "  <t:point>" > fileName;
  print "    <t:CountPoint rdf:about=\"http://transport.data.gov.uk/0/id/traffic-count-point/" cp "\">" > fileName;
  print "      <t:count rdf:resource=\"" base "/motorbike\" />" > fileName;
  print "    </t:CountPoint>" > fileName;
  print "  </t:point>" > fileName;
  print "  <t:hour rdf:datatype=\"http://www.w3.org/2001/XMLSchema#dateTime\">" date "T" hour "</t:hour>" > fileName;
  print "  <t:direction>" direction "</t:direction>" > fileName;
  print "  <t:category rdf:resource=\"http://transport.data.gov.uk/0/category/motorbike\" />" > fileName;
  print "  <rdf:value  rdf:datatype=\"http://www.w3.org/2001/XMLSchema#integer\">" motorbikes "</rdf:value>" > fileName;
  print "</t:Count>" > fileName;
  ...
}
END {
  print "</rdf:RDF>" > fileName; 
  close(fileName);
}

This also generates a shall script that includes the curl instructions to upload the files.

The original data contained easing/northing information about each point when generally latitude/longitude is easier for mapping. So I extracted the easting/northings, used the free (Windows only) software available via the Ordnance Survey to turn these into latitude/longitude — there is a web service to do the same, but you can only do 200 coordinates at a time — converted those into decimals, then RDF, and uploaded them.

The PHP scripts that serve the data as linked data are exactly what I’ve shown before. I amended the .htaccess file to redirect to an appropriate PHP script like this:

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d

  RewriteRule ^id/(.+)$  id.php [L]

  RewriteCond %{REQUEST_URI} !\.php
  RewriteRule ^([^/]+)(/.+)? $1.php$2 [L,QSA]
</IfModule>

and created PHP scripts for each of the types of data being published. For example, region.php is:

<?php
  include "utils.php";
  proxy('http://geo.data.gov.uk/0/ontology/geo#Region', 50);
?>

And there we have it. Linked traffic count data on the web.

(And because this is all published through Talis, there’s also a SPARQL endpoint that you could use to run queries and create visualisations. Knock yourself out.)

Please take a look and comment on what we’ve done. What’s your opinion of the URI scheme? Is it useful to be able to access the data as linked data? Which other formats would you like to see?

Opaque URIs != Unreadable URIs

Jeni — Sat, 25 Jul 2009 21:41:34 +0100

I’ve been talking about URIs a lot recently. One of the things that has bothered me about some of the conversations is the conflation of the concepts of “opaque URIs” and “non-human-readable URIs”. This is my argument for keeping the concepts separate.

The opacity of URIs is an important axiom in web architecture. It states that web applications must not try to pick apart URIs in order to work out information from them. Applications must not, for example, use the fact that a URI has .html at the end to infer that it resolves to an HTML document. It’s closely related to hypertext as engine of application state, in that opaque URIs should not be generated by web applications either: they must be discovered through links and the submission of forms.

But this has nothing to do with readability or hackability, both of which are extremely important for human users. Readable URIs help human users understand something about the resource that the URI is pointing to. Hackable URIs (by which I mean ones that people might manipulate by altering or removing portions of the path or query) enable human users to locate other resources that they might be interested in.

Before I go further, a couple of caveats:

I am not saying that every URI must contain a natural language identifier. An example is the URI for a school, which could include:

the name of the school
the unique reference number for the school
the record number for the school in the database that is being published on the web

Using the name of the school, as I’ve discussed, is probably a bad idea because of its lack of longevity. Using the record number for the school within the particular database that’s being published is entirely non-human-readable because there is simply no way of finding out what that would be for a given school. The unique reference number for the school, on the other hand, may be an obscure series of digits, but it is a meaningful one which renders the URI readable and hackable.

There are also times when uniquely identifying a resource using natural identifiers within the URI leads to incredibly long and complex URIs, in which case the ‘human readable’ version isn’t actually human readable. Introducing non-human-readable components is then the only option.

Back to my argument:

Why should URIs support humans doing things that applications must not? Because humans are intelligent. When humans hack a URI, they are aware that they are making a guess, taking a chance and might or might not end up at something useful. If they get a 404, or even more importantly if they get to information about something that they weren’t expecting, they are intelligent enough to recognise that the chance they took didn’t pay off. Applications aren’t intelligent. They can’t tell the difference between a right guess and a wrong guess, so it’s best not to let them guess at all.

Let me give an example. Let’s say that I’m creating a URI for a particular house. Here are two possible URIs:

http://id.example.org/house/NG9_3HZ/4
http://id.example.org/house/0aef0218

The first is readable and hackable. A human could change the house number or the postcode. They could remove the house number and expect a list of houses within the postcode. The second is not readable or hackable: there is no way to know what you would get if you changed the identifier within the URI.

Now it is true that an application accessing a site that used the URIs like the first could create those URIs programmatically whereas it couldn’t (perhaps) create a URI like the second. But if it did create the URIs programmatically it would be the fault of the application, not the fault of the URI.

As publishers, it is our responsibility to provide humans URIs that are meaningful and hackable, and to provide applications with the means of creating or identifying these URIs through forms and links. But it is not our responsibility to prevent applications from doing things that they should not do by deliberately obfuscating our URIs.

Versioning URIs

Jeni — Wed, 22 Jul 2009 23:16:20 +0100

Yesterday I went along to a workshop on developing URI guidelines for the UK public sector. Because of the current drive to get more UK public sector information online, and the fact that we have Tim Berners-Lee on board, there’s a growing recognition of the fact that we need URIs for the real-world and conceptual things that we talk about in the public sector: schools, roads, hospitals, services, councils, and so on.

One of the particular points of contention at the meeting was whether URIs for non-information resources (ie for real-world and conceptual things) should contain dates or version numbers, or not.

Let’s get some of the argument out of the way first. We are not talking about documents here. Documents will almost always have multiple versions, and if you care at all about maintaining a historical record you will want to refer to the previous version of a document. So dates or version numbers within URIs that refer to documents are often a really good idea. Even better if you have one URI without a date that consistently redirects (through a 307 Temporary Redirect) to the current version of the document.

Documents (that people read) are just one form of “information resource”: things that are information and therefore can be transmitted electronically. Other things in the world are “non-information resources”: things that are more than simple information and therefore cannot be transmitted electronically, such as schools, roads, hospitals and so on. A lot of things that we want to talk about (make RDF assertions about) are non-information resources. We give them URIs to name them, so that we can talk about them unambiguously, and we give them HTTP URIs so that we have a way of finding information resources (documents) that give us information about them.

Does the information that you get when you resolve a non-information resource URI change? Absolutely. A request to a non-information resource URI will respond with a 303 See Other that redirects to an information resource (probably without a version number) that itself redirects (307 Temporary Redirect) to a URI for a particular version of information about the resource. For example an identifier that means a particular school such as:

http://id.example.org/education/school/78

can 303 redirect to the current version of a document that contains information about that school, such as:

http://www.example.org/education/school/78

which will 307 redirect to a particular version of information about that school, such as:

http://www.example.org/education/school/78/2008-09-01

The date is in the URI for the information resource (the information about the school), and therefore it doesn’t need to be in the URI for the non-information resource (the school).

OK, but say that the identifier for a school changes over time. Let’s say that you’ve designed your URIs for schools like:

http://id.example.org/school/bracknell-forest/broadmoor-primary

and the name of the school changes. Now the above identifier isn’t applicable any more, and any RDF statements out there on the web that have used this identifier are now talking about something that no longer exists. How do you deal with this?

Well, the first rule is that non-information resource URIs must not include information that is likely to change. That’s why a lot of URIs contain numbers rather than names. So we shouldn’t have included the name of the school in the URI? OK, we’ll use a number instead:

http://id.example.org/school/bracknell-forest/78

Hang on. Bracknell Forest is a council, and historically it’s been known for councils to change, either in their boundaries (which would mean that a school would move council) or in its name, or they are merged, or… well, there are lots of things that could happen to a council. So in the face of all these possibilities, and given that we no longer need the council name to disambiguate the school name (because we have a number instead), we can employ a second rule: non-information resource URIs must not include unnecessary hierarchy. We can eliminate part of the path and still identify the school:

http://id.example.org/school/78

And so we come to the final thing that could change: “school”. Now surely, you might say, the concept of a school cannot change. And maybe you’re right, maybe it won’t. On the other hand, in the UK we have in the past had things called polytechnics), which are now known as universities, so the types of educational establishments that we have do change over time.

We could do a bunch of things to help prevent a conceptual change like this from requiring a change to the URI:

we keep the number of concepts named within the URI to a minimum (eg don’t have both ‘education’ and ‘school’)
we use wide terms rather than narrow terms (eg use a generic ‘school’ rather than having separate ‘grammar-school’, ‘primary-school’ and so on)
we could change the term ‘school’ to a code (eg use ‘C3X0’ instead of ‘school’), but I don’t think this will help: you’ll still have problems if ‘C3X0’ and ‘F9R2’ mean the same thing in the future, whatever they’re called.
we could eliminate the concept term from the URI altogether, and label everything under one flat naming scheme, using something that has billions and billions of possible combinations. I know, a UUID! No, I’m not serious.

And so we come to the question of versioning the URIs themselves. This is what Tim Berners-Lee says in Cool URIs don’t change:

I’ll go into this danger in more detail as it is one of the more difficult things to avoid. Typically, topics end up in URIs when you classify your documents according to a breakdown of the work you are doing. That breakdown will change. Names for areas will change. At W3C we wanted to change “MarkUp” to “Markup” and then to “HTML” to reflect the actual content of the section. Also, beware that this is often a flat name space. In 100 years are you sure you won’t want to reuse anything? We wanted to reuse “History” and “Stylesheets” for example in our short life.

This is a tempting way of organizing a web site - and indeed a tempting way of organizing anything, including the whole web. It is a great medium term solution but has serious drawbacks in the long term

Part of the reasons for this lie in the philosophy of meaning. every term in the language it a potential clustering subject, and each person can have a different idea of what it means. Because the relationships between subjects are web-like rather than tree-like, even for people who agree on a web may pick a different tree representation. These are my (oft repeated) general comments on the dangers of hierarchical classification as a general solution.

Effectively, when you use a topic name in a URI you are binding yourself to some classification. You may in the future prefer a different one. Then, the URI will be liable to break.

A reason for using a topic area as part of the URI is that responsibility for sub-parts of a URI space is typically delegated, and then you need a name for the organizational body - the subdivision or group or whatever - which has responsibility for that sub-space. This is binding your URIs to the organizational structure. It is typically safe only when protected by a date further up the URI (to the left of it): 1998/pics can be taken to mean for your server “what we meant in 1998 by pics”, rather than “what in 1998 we did with what we now refer to as pics.”

Let’s spell out the danger with some examples. Let’s say that in 20 year’s time, nurseries and primary schools merge into ‘schools’ and secondary schools, sixth-form colleges and universities merge into ‘academies’. A particular primary school currently known as:

http://id.example.org/school/78

will continue to be known by that URI. A particular university currently known as:

http://id.example.org/university/307

is now known as:

http://id.example.org/academy/79

To support these changes, we have to set up some 301 Moved Permanently redirects; http://id.example.org/university/307 has to redirect to http://id.example.org/academy/79. The RDF found at the end of the new URIs has to include owl:sameAs triples that link the new URIs back to the old ones, to indicate they are talking about the same institution:

<http://id.example.org/academy/79> owl:sameAs <http://id.example.org/university/307>

or this would be derived from the 301 response.

Similar changes may or may not happen within the RDF hosted elsewhere that talks about these institutions. Since it can be discovered that they are identical, there’s no real reason for anyone to start using the new URIs unless they want to.

Then 30 years later, the government of the time decide to create a new kind of institution which they call a ‘university’. The university of 50 years hence isn’t actually the same as the ‘university’ as we mean it — they are virtual meeting places for independent researchers, each centered on a particular topic of study rather than a physical location — but they need URIs. And since they are called ‘university’ that is the name that should be used in the URI. Now someone mints the URI:

http://id.example.org/university/307

But disaster! This University 307 is not at all the same as the old University 307, now known as Academy 79. The same URI has been used for two different things. Redirections halt, graphs are smushed, distinctions are lost and fallacies haunt the web.

TimBL’s solution to this possibility is for every URI that includes a topic to include the year in which the topic was minted. So we would have:

http://id.example.org/2009/school/78

that remains the same, and then:

http://id.example.org/2009/university/307

redirecting to:

http://id.example.org/2029/academy/79

and the introduction of:

http://id.example.org/2059/university/307

which can be guaranteed to be distinct from http://id.example.org/2009/university/307.

This, to me, is the crux of the argument for including a version inside the URIs that you use for non-information resources. It means that you can reuse old terms with new meanings within URIs without breaking the web.

On the other hand, many people, myself among them, really dislike the use of years or version numbers within URIs for non-information resources (unless, I should say, they are used as part of the identification of the resource). I think there are four main reasons:

they are additional cruft that add to the length of a URI but provide no information about the thing being identified
they can give a misleading impression about the relevance of a concept; for example FOAF is stuck at version 0.1 (http://xmlns.com/foaf/0.1/) despite being widely used, while http://www.w3.org/1998/Math/MathML is feeling distinctly old (in internet time) despite being under active development
it leads to a proliferation of URIs and creates additional work for people who want to keep their URIs up to date, even when the concepts themselves don’t change (such as for the primary school’s URI above)

In essence, the likelihood of a term being reused with a different meaning seems low enough that the cost (in readability, understandability and maintainability) of supporting URIs that contain versions or years doesn’t seem worthwhile. We can keep the likelihood low by using terms that are unlikely to change their meaning (particularly avoiding those that have more than one meaning) and by disambiguating them (for example by using ‘train-station’ rather than just ‘station’).

There is also, perhaps, a middle way here that can keep the majority of URIs clean without leading to overlapping names. That’s to start with a URI scheme that does not include a version number or year, and only to start introducing them when it becomes necessary due to the reuse of previous terms. In the example above, in 2059 we might have:

http://id.example.org/school/78
http://id.example.org/academy/79
http://id.example.org/university2.0/307

In other words, we make a decision now that our future selves will have to act upon. All we have to worry about is our future selves caring as much about persisting historical URIs as we do about persisting our current ones.

What do you think? Should versioning be avoided in URIs at all costs, or always be included just in case? Are there other arguments for or against including versions or years in URIs? What other design considerations are there that help prevent changes to URIs over (long periods of) time?