rdf

Metadata about RDF triples: reification and Linked Data

Jeni — Fri, 11 Apr 2008 21:59:07 +0100

Those of you who have been following this blog will know that I’ve been thinking recently about how to handle uncertainty related to RDF triples (specifically in the context of a genealogical web app). Certainty isn’t the only kind of metadata-about-triples that you’d want to keep in an app like this. We need to know things like:

who made the statement
when the statement was made
what evidence that led to the statement being made
licensing information about the reuse of the statement
(if we go with the rating idea) what ratings the statement has been given
(if we allow editing of statements) what changes have been made to the statement over time

and so on. In short, all the metadata that you’d want to associate with resources you’d also want to associate with statements.

I’d anticipated using reification to associate metadata with statements. Something like this

<rdf:Statement rdf:about="#statement1">
  <rdf:subject rdf:resource="/people/CharlesDarwin" />
  <rdf:predicate rdf:resource="/ontology/event-roles/passenger" />
  <rdf:object rdf:resource="/events/BeagleVoyage" />
  <dc:creator rdf:resource="/users/JeniT" />
  <dc:date rdf:datatype="xsd:date">2008-04-11</dc:date>
  <g:certainty rdf:datatype="xsd:decimal">1.0</g:certainty>
  ...
</rdf:Statement>

or using rdf:ID, although this does limit the URI of our statements to hash-URIs:

<rdf:Description about="/people/CharlesDarwin">
  <r:passenger rdf:ID="#statement1" rdf:resource="/events/BeagleVoyage" /> 
</rdf:Description>
<rdf:Description about="#statement1">
  <dc:creator rdf:resource="/users/JeniT" />
  <dc:date rdf:datatype="xsd:date">2008-04-11</dc:date>
  <g:certainty rdf:datatype="xsd:decimal">1.0</g:certainty>
</rdf:Description>

(Please feel free to correct my RDF, RDF-folks!)

We can embed this information into our web pages using RDFa:

<div about="#statement1" instanceof="rdf:Statement">
  <p class="statement">
    <a rel="rdf:subject" href="/people/CharlesDarwin">
      Charles Darwin
    </a>
    was a
    <a rel="rdf:predicate" href="/ontologies/event-roles/passenger">
      passenger
    </a>
    on the
    <a rel="rdf:object" href="/events/BeagleVoyage">
      <span about="/people/CharlesDarwin" 
            rel="r:passenger" 
            resource="/events/BeagleVoyage">
        Beagle Voyage
      </span>
    </a>
  </p>
  <dl class="metadata">
    <dt>Author:</dt>
    <dd>
      <a rel="dc:creator" href="/users/JeniT">
        Jeni Tennison
      </a>
    </dd>
    <dt>Date:</dt>
    <dd property="dc:date" datatype="xsd:date" 
        content="2008-04-11">
      11 Apr, 2008
    </dd>
    <dt>Certainty:</dt>
    <dd property="b:certainty" datatype="xsd:decimal"
        content="1.0">
      <img src="stars5.gif" alt="five stars" />
    </dd>
  </dl>
</div>

Note that I’ve incorporated both the reified statement and the statement itself into the RDFa. If I’m correct in my mental parsing of RDFa, I think this leads to the set of triples from the RDF/XML in the above examples plus the triple:

</people/CharlesDarwin> r:passenger </events/BeagleVoyage> .

But then the other day, I was reading the tutorial How to publish Linked Data on the Web, which says

We discourage the use of RDF reification as the semantics of reification are unclear and as reified statements are rather cumbersome to query with the SPARQL query language. Metadata can be attached to the information resource instead, as explained in Section 5.

Jumping to Section 5, I find

Metadata: The representation should contain any metadata you want to attach to your published data, such as a URI identifying the author and licensing information. These should be recorded as RDF descriptions of the information resource that describes a non-information resource; that is, the subject of the RDF triples should be the URI of the information resource. Attaching meta-information to that information resource, rather than attaching it to the described resource itself or to specific RDF statements about the resource (as with RDF reification) plays nicely together with using Named Graphs and the SPARQL query language in Linked Data client applications…

There are some examples of what this looks like within the tutorial. The first is an “authoritative description” found at http://dbpedia.org/data/Alec_Empire after a 303 redirection from http://dbpedia.org/resource/Alec_Empire.

# Metadata and Licensing Information
<http://dbpedia.org/data/Alec_Empire>
    rdfs:label "RDF description of Alec Empire" ;
    rdf:type foaf:Document ;
    dc:publisher <http://dbpedia.org/resource/DBpedia> ;
    dc:date "2007-07-13"^^xsd:date ;
    dc:rights <http://en.wikipedia.org/wiki/WP:GFDL> .

# The description
<http://dbpedia.org/resource/Alec_Empire> 
    foaf:name "Empire, Alec" ;
    rdf:type foaf:Person ;
    rdf:type <http://dbpedia.org/class/yago/musician> ;
    rdfs:comment
        "Alec Empire (born May 2, 1972) is a German musician who is ..."@en ;
    rdfs:comment
        "Alec Empire (eigentlich Alexander Wilke) ist ein deutscher Musiker. ..."@de ;
    dbpedia:genre <http://dbpedia.org/resource/Techno> ;
    dbpedia:associatedActs 
      <http://dbpedia.org/resource/Atari_Teenage_Riot> ;
    foaf:page <http://en.wikipedia.org/wiki/Alec_Empire> ;
    foaf:page <http://dbpedia.org/page/Alec_Empire> ; 
    rdfs:isDefinedBy <http://dbpedia.org/data/Alec_Empire> ;
    owl:sameAs <http://zitgist.com/music/artist/d71ba53b-23b0-4870-a429-cce6f345763b> .

The second is a non-authoritative description found at http://sites.wiwiss.fu-berlin.de/suhl/bizer/pub/LinkedDataTutorial/ChrisAboutRichard:

# Metadata and Licensing Information
<>
    rdf:type foaf:Document ;
    dc:author <http://www.bizer.de#chris> ;
    dc:date "2007-07-13"^^xsd:date ;
    cc:license <http://web.resource.org/cc/PublicDomain> .

# The description
<http://richard.cyganiak.de/foaf.rdf#cygri> 
    foaf:name "Richard Cyganiak" ;
    foaf:topic_interest <http://dbpedia.org/resource/Category:Databases> ;
    foaf:topic_interest <http://dbpedia.org/resource/MacBook_Pro> ;
    rdfs:isDefinedBy <http://richard.cyganiak.de/foaf.rdf> ;
    rdf:seeAlso <> .

Note that rdfs:isDefinedBy does not necessarily point to the data you get when you retrieve the resource, but to an (presumably there can be more than one) authoritative description of the resource. It’s also associated with a particular resource rather than a particular statement.

To know which metadata applies to a particular statement, an application must know where it got the statement from. In effect, a statement here has four parts: subject, property, object and location (with the possibility that multiple statements with the same subject, property and object might have different locations and therefore different metadata). This is similar to assigning an ID to a statement, as with rdf:ID, but restricts the statement’s identifier to being the location where it was found.

So what does that mean for the genealogical web app? Well, in the app we’re going to find any given statement by a particular user quoted on lots of pages. I was intending to RDFa them all but that would mean lots of duplicate statements from different locations, potentially bloating applications that were harvesting the data.

I can’t work out whether I like or loathe the Linked Data concept of associating metadata with the document in which you find triples. In some ways it seems very natural — look for information about a resource at the URI for the resouce — but the metadata mechanisms restrict where you can place statements on the web (or at least assign semantics to their location which aren’t necessarily intended), and that seems like a Bad Thing. On the other hand, perhaps I’m just being overly influenced by the desire to use RDFa, which does lead one to want to mark up data wherever it appears.

I’d welcome any advice.

RDF and XML Q&A: Which should I use?

Jeni — Sun, 17 Feb 2008 20:10:12 +0000

Another question to answer:

I’ve been reading about RDF, and I’m not sure in what situations it is more appropriate to use RDF over straight XML. I usually see RDF expressed as XML, but sometimes I see it written as language-independent functions (or methods).

Part of me is wondering if RDF is more appropriate for this project. What might the benefits be? And if it is, how difficult it would be to refactor it.

(Note that the person asking the question is talking about a small data-oriented project.) There’s a huge amount that could be said about this, so I might well post about some of it again. Here, I’m going to cut to the chase. This is what I’d recommend:

Model your application in RDF terms: Create a description of what classes of resources your application needs to deal with, and which properties link those together. You can call this description a RDF schema or conceptual model or ontology, depending on how impressive you want to sound. This modelling activity is useful in itself, largely because it helps you understand what information you’re dealing with and how it fits together.
Create a markup language that can be mapped to RDF: An XML version of your data allows you to make your data more generally available and reusable than locking it away in a triple store. Do one of the following:
- Define a subset of RDF/XML for your application: The full flexibility of RDF/XML is complicated to handle for plain XML processors, so subset it to, for example, always used typed elements (such as <my:Course>) rather than rdf:type properties, and to use referencing or nesting in a consistent way.
- Design markup languages that use RDFa attributes to reflect the semantics of the data: This gives you a standard way of mapping your markup language into RDF triples without having to adopt the “striped” design of RDF/XML in your markup language. A lot of the attributes can be defaulted to leave the markup language fairly streamlined.
- Design markup languages exactly as you like, and define GRDDL mappings from them into RDF/XML: This gives you the most flexibility in your markup language design (though not complete flexibility — you still need to be able to identify the statements that you want to make from the XML), at the expense of having to write some XSLT.

The point of doing this is to put you in a position where you can just use XML if you want, but you also have the flexibility of using RDF either now or in the future.

The benefits of using RDF are partly to do with the ease with which you can do certain kinds of processing (specifically combining “facts” together to draw conclusions) and partly to do with the potential of reuse of your data. In the same way that XML gives people a common syntax and thus aids interchange of information, RDF allows others to draw some conclusions (more than they would with a random mess of elements and attributes) about what your data means.

I don’t think that using RDF triple stores, SPARQL and all that jazz gives you a great return for a small-scale, personal project — you’re better off sticking to flat files and some XSLT — but it doesn’t hurt to build in some of the formality of RDF anyway.

Web 2.0 project: RDF and uncertainty

Jeni — Fri, 21 Dec 2007 22:03:51 +0000

I’ve been thinking a bit recently about how to deal with certainty in our Genealogical Web 2.0 application. We’ve come round to using an RDF model to represent what the Gentech data model calls “assertions”; assertions such as “Charles Darwin was a passenger on the Beagle Voyage” are represented as an RDF Statement in which (a resource representing) “Charles Darwin” is the subject, (a resource representing) “Beagle Voyage” is the object, and “was a passenger on” is the predicate/property.

All the statements in the genealogical application should be based on some source of information, either an external piece of evidence (such as a marriage certificate) or by combining existing statements. Either way, there’s certain metadata that we want to store about it, such as

who created the statement
when it was made
the date(s) when the statement was true
the certainty in the statement

The certainty factor is interesting. For statements based directly on evidence, there are three factors that come into play:

the reliability of the evidence itself; for example, a marriage certificate is more reliable than a diary entry for a wedding
the certainty the user has in drawing their conclusion based on the evidence; for example, you would be more certain in the statement that the groom named on a marriage certificate is a man than in the statement that the witness named on a marriage certificate is a friend of the groom
the reliability of the user who has made the statement: an expert in family history is likely to draw more accurate conclusions than someone who has only just started

So now the question is how to assess these factors. The usual Web 2.0 method is to use ratings. We could get users to rate each other to provide the third score. We could then get users to rate the reliability of particular pieces of evidence, modify that score based on the users’ reliability, and aggregate those scores.

The final certainty of the statement would be a combination of this score for evidence reliability and ratings from multiple users, again weighted according to the users’ reliability.

Converting (people) to RDF

Jeni — Thu, 15 Nov 2007 20:04:33 +0000

As I’ve mentioned before, I’ve been a RDF sceptic for a long time. Perhaps it’s precisely because of my knowledge engineering background: in my experience, the field is about equal parts academic optimism, sales-related exaggeration and plain old information management. In other (un-minced) words, unrealistic aims with unproven technologies that are sold as being much cleverer (and more innovative) than they are. It’s not just RDF, I should say, but the whole Semantic Web pitch (typified for me by the idea of halting global terrorism using the power of Topic Maps) that seemed ludicrous to me.

Time moves on, and I might be changing my mind.

Putting the whole over-selling thing to one side, one underlying reason for my scepticism was that it seemed that the Semantic Web depended on people doing things in one way: using big standard ontologies with RDF/XML representations. That doesn’t seem to me to fit with the web. The power of the web comes from people (largely) being able to do their own thing, in their own time, with opportunistic linking between sets of information. If you want to use the web, you have to work with the loose coupling.

But RDF is becoming more web-friendly. GRDDL and RDFa both provide mechanisms for providing RDF triples whilst retaining your own markup language. GRDDL does it by pointing at some XSLT that will turn your markup into RDF/XML, while RDFa provides a set of conventions that you can use in your markup that allow it to be directly interpreted as RDF by RDFa processors. Either way, it means you don’t have to adopt the striped syntax pattern in your markup, or even use the RDF namespace. The RDF data model always was theoretically distinct from the RDF/XML syntax; these new technologies (plus simple notations such as Turtle) place the emphasis back on the model.

Also encouraging is that OWL specifically supports linking between ontologies, allowing you to indicate that classes, properties and individuals from different ontologies are equivalent. This means you are free to create your own local ontology, as am I, and either of us (or a third party) can later link the two ontologies together, along with any related triples.

However, I was chatting to my friend Louise the other day about RDF, and asked her whether she’d be using it in her next project. Her basic argument against was simply that it wasn’t worth it. And this is the second underlying reason for my scepticism: does RDF give a big enough win to justify the effort required to use it? The “market” seems to say not. RDF has been around for years; if it was going to become an established technology, surely it would have by now. When I look at the web today, it’s pretty hard to find RDF. Where semantic information is available, it’s usually provided in a JSON or XML format. Or of course they’re using microformats in their pages. Why go to the bother of defining an ontology and providing a GRDDL transformation, or adopting the rigour of RDFa, when these technologies get the job done?

It should be that mashing up RDF is easier than mashing up random JSON/XML. Just a few “this is equivalent to that” assertions and you’re good to go. So show me the places (outside academia) where RDF-based mash-ups are being made. Show me the interfaces that are allowing people in the real world to use RDF to pull together information from diverse sources, get better overviews, draw more accurate conclusions.

I want to believe. Show me the uses of RDF that will convert the sceptic in me.

RDFa, sure, but now?

Jeni — Sun, 19 Aug 2007 21:34:35 +0100

I think RDFa has a lot of things going for it. I’ve been a big Semantic Web sceptic for a long time, but getting semantic information embedded into web pages is cool, and RDFa is that much more rigorous than microformats. Sure, it’s a bit more complicated too, but I’m not afraid of namespaces!

The problem is that I’m supposed to be assessing the introduction of RDFa into a biggish, important, real-world website. It’s a website where every change has to go through a Process. There are project managers, development managers, product managers. There are functional specifications, technical specifications, and rollout documents. There are unit tests, peer reviews and user acceptance tests. The database is huge; the HTML is pre-generated.

In short, every change takes time and costs money. I’m just not sure it’s a good idea to introduce a technology that only exists in a Working Draft of a Primer now, when tracking that technology could be very costly.

And if we used RDFa, we wouldn’t be using XHTML 1.0 any more; we’d be using XHTML 2.0. If we want to have valid pages, we’d have to reference a non-existent DTD, or make our own. And tracking XHTML 2.0 is even more of a risk, it seems to me.

So my current thinking is that in this case, RDFa just isn’t practical. Better to use GRDDL with something less revolutionary, like eRDF or microformats or plain-old custom class names.

Anyone want to persuade me otherwise?

Things that make me scream: RDF "QNames"

Jeni — Wed, 15 Aug 2007 22:38:28 +0100

Having avoided RDF like the veritable plague for years, I have been forced to look at it properly for my latest “put RDFa on our web pages” project. So the other day I came across this weirdness surrounding “QNames” in Turtle, SPARQL and RDFa (and so on)…

As we all know, RDF is about making statements about resources, and resources are identified by URI. And the predicates/properties that you use to make statements about resources are also identified by URI. So I can say things like (in Turtle syntax):

<http://www.lmnl.org/wiki/Creole>
  <http://purl.org/dc/elements/1.1/creator>
    [<http://xmlns.com/foaf/0.1/weblog>
     <http://www.jenitennison.com/blog>] .

and it means that the creator (as defined by Dublin Core) of the page <http://www.lmnl.org/wiki/Creole> is someone whose weblog (as defined by FOAF) is <http://www.jenitennison.com/blog/>.

The trouble with URIs is that they’re so long. Turtle, SPARQL and RDFa, and all of these technologies want to make it easy to write RDF, so they want to provide a shorthand. The shorthand they provide looks similar to, and is even sometimes called, a “QName”.

Here’s how “QNames” work in these RDF technologies. You associate a prefix with a URI like so (this from Turtle):

@prefix dc: <http://purl.org/dc/elements/1.1/> .

Then when you write the prefix followed by a :, followed by a local part, it’s equivalent to concatenating the URI with the local part; the result is the URI represented by the “QName”. So having defined the dc prefix as above, I can use dc:creator in place of the URI <http://purl.org/dc/elements/1.1/creator>.

Seems simple enough, right? I mean, I know how QNames work in XML, and these look the same, just with some behind-the-scenes assumptions about how to resolve QNames to get to further information about them.

My first double-take was when I wanted to add a datatype to a literal. I wanted to indicate that the value was a date, which you can do in Turtle or SPARQL with the syntax:

"2007-08-13"^^xs:date

I can write the namespace for XML Schema on auto-pilot, so I started writing it <http://www.w3.org/2001/XMLSchema>, then wondered whether they just used the datatypes URI so went to check. What URI is used in the examples? <http://www.w3.org/2001/XMLSchema#>. Huh? What’s that # doing there? Well, it’s because the full URI is then <http://www.w3.org/2001/XMLSchema#date>, which looks a whole lot better than <http://www.w3.org/2001/XMLSchemadate>. It doesn’t matter that <http://www.w3.org/2001/XMLSchema#date> doesn’t resolve to anything (or does it?), so long as we all use the same URI to mean date-as-defined-in-XML-Schema. The small matter that the RDF “namespace” for XML Schema is different from the normal XML namespace for XML Schema is by-the-by. Boggle.

My second double-take was when I wanted to write an example that used numbers as identifiers. My last post was about URL design, and used the URL <http://www.statutelaw.gov.uk/legislation/3032571> as an illustration. Now say I want to use a QName to represent that URI. I can do

@prefix leg: <http://www.statutelaw.gov.uk/legislation/> .

but if I then do leg:3032571 that’s not a legal (XML) QName: the local part isn’t an XML Name because it begins with a number. Turtle doesn’t allow it, SPARQL kinda does (it’s an “at risk feature”), and who knows what RDFa will do (there’s not enough of a spec to tell as yet).

Then there’s the weirdnesses of “QNames” with no prefix, which still have a colon. And QNames with an underscore as the prefix, which of course mean a blank node with a specified ID, but let’s leave that to one side.

Really, the whole mess started with RDF/XML which allows you to give elements the “QName” equivalent of the class that you’re talking about. So <rdf:Description> can be replaced by <foaf:Person>. Indeed, properties pretty much have to be represented by elements whose name is equivalent to the property URI in RDF/XML. That doesn’t usually matter so much because, unlike instances, which could be pretty much any URI, properties and classes are likely to have sensible names (rdf:_1, rdf:_2 and so on being notable, but illustrative, exceptions: it really shouldn’t be necessary to add underscores just to satisfy the QName naming rules which don’t apply in the abstract RDF model).

Fundamentally, using “QNames” as abbreviations for URIs is a bad idea. QNames have a number of restrictions on them because they are built to be legal XML Names: the kinds of things that you can call elements and attributes. URIs don’t have these restrictions: it’s perfectly possible for the last part of a URI to consist purely of numbers, or to have a slash at the end, or even to have request parameters. Fair enough that meaningful QNames can be used for some URIs, but if you can’t use them properly for all URIs, then there has to be a better way.

The way that you usually create short URIs is to use a relative URI, which is resolved against a base URI to produce an absolute URI. And that’s the mechanism that should be used in RDF. The only difficulty is that in most RDF documents, you’ll have lots of URIs with completely different base URIs, so having just one base URI (like normal) doesn’t cut it.

You need to be able to associate shorthands (I won’t call them prefixes, because doing so is confusing) with base URIs — a @base directive will do — and have a simple syntax for resolving relative URIs against a given base. You could use a shorthand:<relativeURI> syntax, I think, and allow the <>s to be dropped if the relative URI happens to be a legal Name. So with

@base leg: <http://www.statutelaw.gov.uk/legislation/> .

you could use leg:<3032571> to mean the URI <http://www.statutelaw.gov.uk/legislation/3032571> or leg:<wine/2007/-/> to mean the URI <http://www.statutelaw.gov.uk/legislation/wine/2007/-/>.

SPARQL already has a BASE directive to define a single base URI that’s used when resolving relative URIs in URI references; really all I’m suggesting is allowing several base URIs to operate at the same time (using shorthands to distinguish between them), and dropping PREFIX. Turtle doesn’t have a @base directive, though there’s a suggestion that one might be added, so there the changes are a little more major.

In RDFa, “QNames” are used in attribute values to indicate class URIs, property URIs, and datatype URIs. And normal namespace declarations are used to make the prefixes, which means training yourself to write

xmlns:xs="http://www.w3.org/2001/XMLSchema#"

when you’re writing RDFa, while everywhere else (schemas, stylesheets, etc.) you write

xmlns:xs="http://www.w3.org/2001/XMLSchema"

XML namespace declarations really shouldn’t be used here. Instead, you could extend the <base> element-as-was to provide bindings

<base short="xs" href="http://www.w3.org/2001/XMLSchema#" />
<base short="leg" href="http://www.statutelaw.gov.uk/legislation/" />

and then use those bindings when interpreting the attribute values (the shorthand:<relativeURI> syntax won’t work here, because of having to escape the <s in attribute values, but you get the idea).

Meanwhile, if you have free reign over your URL space and you’re thinking of using RDF, URLs should be designed such that the last part (be it fragment or path segment) is a legal XML local name. Doing so will make it a lot easier to express statements in Turtle or RDFa and queries in SPARQL. What’s more, the last part of the URI should be the unique identifier for the resource in question; if your URLs look like <http://www.example.com/collection/5326/view> then you should consider dropping the /view part to provide the canonical URI, or swapping the last two parts of the path (oh and adding a prefix to the numeric identifier) to give <http://www.example.com/collection/view/i5326>.

And this is particularly true, indeed fairly essential, for any URLs that you might want to use to represent classes or properties.