rdfa

Microdata and RDFa Living Together in Harmony

Jeni — Sat, 20 Aug 2011 16:39:11 +0000

One of the options that the TAG put forward when it asked the W3C to put together task force on embedded data in HTML was the co-existence of RDFa and microdata. If that’s what we’re headed for, what might make things easier for consumers and publishers who have to live in that world?

In a situation where there are two competing standards, I think that developers — both on the publication and consumption sides — are going to want to hedge their bets. They will want to avoid being tied to one syntax in case it turns out that that syntax isn’t supported by the majority of publishers/consumers in the long term and they have to switch.

Publishers like us at legislation.gov.uk who are aiming to share their data to whoever is interested in it (rather than having a particular consumer in mind) are also likely to want to publish in both microdata and RDFa, rather than force potential consumers to adopt a particular processing model, and will therefore need to mix the syntaxes within their pages.

(Of course developers might just avoid embedded data altogether while they wait to see what happens, but let’s assume that they want to press ahead regardless of the lack of consensus from the standardistas.)

I’ve therefore embarked on a task of trying:

to identify the differences in approach and functionality of the two languages, which should help developers choose between them
to identify any guidelines for developers of vocabularies for use with both languages
to identify a subset of functionality that is common between the two languages, which developers might want to stick to to make switching and mixing easier
to identify mapping rules that might be applied to automatically or manually map from one language to another if the simple subset is used

I’ve done this by looking at converting microdata examples to RDFa and vice versa, and the lessons to be drawn from that exercise. I’ve broken down the result into three posts:

This is the last of these posts. It is probably the only one you will want to read :)

Please treat this as a draft on which I’d welcome comments. I have based what’s written here on the latest specifications of both microdata (in its WHAT WG and W3C variants) and RDFa Core and HTML+RDFa but I haven’t consulted with anyone involved in these efforts and may well have got things wrong. Plus the specs are changing all the time. I have only here considered the syntax of the two languages, not the features such as DOM APIs or drag-and-drop support, where there are also clear differences.

Please add comments if there are things that I’ve missed or got wrong, or just to have your say.

Philip Jägenstedt’s Live Microdata service and Gregg Kellogg’s Distiller service have both proved invaluable for testing — thank you to both for making these services available. I heartily recommend them.

Mapping Rules

The first problem is how to judge equivalence when microdata and RDFa have different data models. Microdata essentially uses a JSON data model: there are objects (items) with properties that have values that are strings, other objects, or arrays of strings or objects or both. RDFa naturally uses a RDF data model: there are resources with properties that have values that are literals (of some datatype or with a language) or other resources.

Underlying both is the same basic entity-attribute-value pattern, but there are various mismatches between the models that make some mappings more complicated than others, or in other cases mean that information is necessarily lost on conversion.

In performing the analysis, I’ve tried to map microdata into sensible RDF and then match that RDF output using RDFa, and to map RDFa into sensible microdata+JSON and then match that microdata+JSON using microdata. The microdata-to-RDF mapping rules that I’ve followed are basically those outlined in my post on Microdata + RDF. To create microdata JSON from RDFa, I’ve used the rule that the URI of the first type of a resource is processed to provide a namespace that is stripped from the URIs of the properties (to create simple names where possible). In addition, when a resource has no properties, it will be represented as a string (URI) value rather than as a nested item.

These rules need to be formalised, obviously, but the basics above work well enough for the examples from the specs.

Mismatched Features

The following features are problematic when mapping from microdata to RDFa or vice versa. I’ve described them roughly in an order from things where it might be relatively easy to address the problem by changing one or other specification, to places where the necessary changes would be difficult to make in the specs, which means that publishers and consumers need to be aware of the issue so that they can make an educated choice about how they proceed.

Local Property Names

Many of the microdata examples involve items with no type and local property names. I’ve assumed in the analysis below that this generates properties whose URI is based on the document in which they are found, but this is not a helpful solution for data sharing: if a whole site uses short property names across its pages, those properties really need to be recognised as being the same across the site for any kind of useful processing to occur.

What microdata actually creates here is a global namespace, shared by everyone, specifically for embedded data. There are three things that could be done at different levels here:

In a mapping from microdata to RDF, any short property names on items that don’t have a type could be assigned to a global namespace (eg http://w3.org/ns/global/). Of course there will be clashes in semantics within this namespace, but that is true in microdata generally and not having to create a new namespace makes the initial experimentation easier for those starting with embedded data. The W3C (or whoever operates the namespace) could operate a wiki at that location that would operate as an informal registry for the property names.
HTML+RDFa could change to use this global namespace as the default vocabulary URI (rather than not having one). This would make it a little easier for people to convert microdata to RDFa: if they don’t use types for their items, there would then be no need for a vocab attribute to be added to the HTML. It also makes it possible to use RDFa in a basic, lightweight way, which might help people get started with it.
Publishers can be advised to use itemtype within their microdata, reusing existing classes or creating their own, if they want to ensure that the embedded data within their pages isn’t misinterpreted by global consumers.

Interpretation of `<time>` Element’s `datetime` Attribute

Interpreting the datetime attribute of the <time> element to supply a value, rather than repeating that value in a content attribute, is ISSUE-97 on RDFa, and hopefully RDFa will be changed to use that value (or the content of the element if there is no datetime attribute), add a seconds component if necessary, and work out an appropriate date/time datatype for it based on its syntax.

Content Overrides

In RDFa, publishers can provide a machine-readable version of the content of an element (or even an entirely different value) using the content attribute. This can only be done for date/times in microdata. The ability to annotate non-date/time content with machine-readable values is a current issue on HTML5. Resolving this in favour of providing such annotation would make using RDFa and microdata in concert, or converting between them, easier, particularly if HTML5 uses the attribute content or RDFa adopts whatever attribute is introduced to HTML5.

`<link>` and `<meta>` Elements in Flow Content

The ability to use <link> and <meta> elements in flow content is only supported in microdata: it’s support that’s added by the microdata specification (in the Editor’s Draft since May 31st; the text allowing this didn’t make it into the Last Call version of the spec), in which it’s limited to <link> and <meta> elements with an itemprop attribute.

It would be possible for the RDFa specification to similarly make the statement that <link> and <meta> elements are allowed in flow content as long as they have particular attributes. This would ease the transition between the two formats, and works a lot better than empty <span> elements which crop up fairly commonly in RDFa content.

(One oddity here is that because date/time values have to be on a <time> element in microdata, publishers cannot replace empty <time> elements with <meta> elements as they might an empty <span>.)

Identifiers without Types

Many of the RDFa examples are of resources that have a URI identifier but for which no type is supplied. Microdata, on the other hand, states that itemid is only allowed on elements that also have an itemtype (and an itemscope). The reason given is because the itemid needs to be interpreted based on the itemtype. This would be understandable if it held a string, but given that the itemid provides a URI it seems a bit strange. Perhaps it’s an attempt to avoid the whole httpRange-14 / ambiguity in URIs issue.

If this restriction remains, the advice to RDFa users who might want to convert to microdata at a future date would be to always provide a type for your (non-blank-node) resources. It may be useful to define a http://w3.org/ns/global/Thing within the vocabulary that I propose above, given that the URI for rdfs:Resource is long and hard to recall.

Built-in Prefixes

The built-in profile for RDFa defines a number of prefixes for vocabularies that are either coined by the W3C or coined elsewhere but in common use on the web. This, coupled with vocab and the ability to directly use URIs in the relevant attributes, means that declaring prefixes within the document is increasingly unnecessary in RDFa.

In contrast, using existing vocabularies, even popular ones, within microdata is relatively difficult, particularly when vocabularies are mixed on the same item.

Most useful for publishers would be if both RDFa and microdata recognised the same set of prefixes. This would reduce the size of microdata created from existing RDFa content as well as making it easier to move between the languages. At the very least, it would be good to have rdf:, rdfs:, xsd: and xhv: built into both.

The list of popular vocabularies is likely to change over time; for example a prefix for the schema.org vocabulary might be useful at some point in the near future. The problem is that publishers and consumers need to be synchronised in their use of prefixes: it’s no good for a publisher to use the prefix sch: if there might be processors for the page that don’t recognise it. Equally, consumers shouldn’t be reliant on a network connection to retrieve the latest set of prefix mappings in order to parse the page. It’s not clear to me how best to manage this evolution, but even a fixed set of prefixes at the point the specs reach Recommendation is more usable than spelling out URIs all the time.

Literals Including Markup

RDFa supports literals that include markup (the innerHTML of an element) as well as those that don’t (the textContent of an element), whereas microdata only supports creating values from particular attributes or the textContent of the element. This makes it hard to create embedded microdata that includes values which contain things like mathematical or chemical formulae, ruby text, or multiple paragraphs.

A solution would be for microdata to introduce an itemhtml (or something) attribute that, when present, indicates that the value of the property should include markup. There is a current issue on microdata to support HTML values.

Itemref

RDFa can support a subset of itemref’s functionality, namely to have properties defined elsewhere in a document be associated with a given resource. What it doesn’t support is the sharing of properties defined in one place by two or more resources.

RDFa could add such support by adding an attribute that mirrors itemref (eg ref, I guess), with the referenced element being processed using the evaluation context inherited by the referencing element (which means that attributes such as vocab would sometimes have a scope that wasn’t based on the document tree). This would make it easier to tackle the use case for itemref using RDFa as well as making it easier to move between or mix RDFa and microdata.

Lists

It is easy for microdata to represent a property with a list of values, and really really hard to do the same in RDFa. This is in part because RDF views lists resources rather than a distinct data type, and in part because RDFa hasn’t added any syntax sugar to make creating rdf:List resources easy. Adding some syntax sugar for lists would make life a lot easier for anyone using RDFa, but especially if they are adapting existing microdata content to RDFa.

Datatypes

Microdata assumes that consumers will convert values to appropriate datatypes based on the property (which they understand) as a separate process after microdata processing, whereas RDFa supports the use of a datatype attribute to explicitly indicate the datatype of each value. This mismatch means that information is lost when RDFa is converted to microdata, and has to be added when microdata is converted to RDFa.

Bringing the languages completely into sync would mean either microdata adding a facility to support (at least some) datatypes, or deprecating the datatype attribute in RDFa. Alternatively, this may simply be an area where the differences in behaviour between the two specifications doesn’t matter because the data models that they use are distinct anyway.

Languages

Languages are similar to datatypes, in that RDF (and hence RDFa) supports annotating strings with the language that they are in whereas microdata doesn’t within its core data model or its JSON serialisation. However, the elements that represent properties within the HTML, used within the DOM API access to microdata, will have a language.

It may be that in practice consumers need to base their microdata processing on the DOM API rather than the core microdata data model or JSON extracted through a standalone process, and thus pick up the language from the property elements, I don’t know. In any case, the microdata JSON serialisation, used for drag-and-drop, is lossy and could be extended to include the language of each value when available, at fairly substantial complexity cost.

For publishers, it doesn’t much matter either way; if they are dealing with multi-lingual text they will want to include a lang attribute in the HTML anyway, regardless of the impact on embedded data.

Multiple Types

RDFa supports having multiple types named in the typeof attribute whereas microdata only supports one type per item. In any mapping from RDFa to microdata, publishers have to choose which type is the primary type for the item and move the others to be expressed via rdf:type properties. Consumers who want to support publishers who might not choose their type as the primary type have to detect items that have the type they are interested in within the rdf:type property as well as those which have the type as the main type. Given that the rdf:type URI is long and (naturally) associated with RDF, it might be better to define a property such as http://w3.org/ns/global/type for this use.

Microdata could be extended to allow multiple values in the itemtype attribute, with the first being used to interpret any properties that aren’t full URIs. This would make it easier for both consumers to detect when a type they were interested in was used and for publishers to use RDFa and microdata in tandem or move between them.

The `src` Attribute

RDFa and microdata interpret the src attribute in opposite ways. In RDFa, it provides the identifier for a new resource (equivalent to itemid in microdata); in microdata, it provides a URL value of a property on elements that support it (equivalent to resource or href in RDFa).

RDFa interprets src in this way to make it easier to make assertions about an image, but it’s of limited effect as even in RDFa its only possible to make three such assertions (through the typeof, rel and property attributes). So, for example, you can specify the type of the image, link to its license and give the name of its creator, with:

<img src="photo1.jpg" typeof="foaf:Image"
  rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
  property="dc:creator" content="Mark Birbeck">

but this won’t help you if you also want to give the title for the image and when it was created (say). At that point, the microdata and RDFa start to look similar:

<div itemscope itemid="photo1.jpg" itemtype="http://xmlns.com/foaf/0.1/Image">
  <link itemprop="license" href="http://creativecommons.org/licenses/by/2.0/">
  <meta itemprop="http://purl.org/dc/terms/creator" content="Mark Birbeck">
  <meta itemprop="http://purl.org/dc/terms/title" content="Picture of Mark">
  <time itemprop="http://purl.org/dc/terms/created" datetime="2009-03-17"></time>
  <img src="photo1.jpg">
</div>

or:

<div about="photo1.jpg" typeof="http://xmlns.com/foaf/0.1/Image">
  <span property="http://purl.org/dc/terms/title" content="Picture of Mark"></span>
  <time property="http://purl.org/dc/terms/created" content="2009-03-17" datatype="xsd:date" datetime="2009-03-17"></time>
  <img src="photo1.jpg" typeof="foaf:Image"
    rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
    property="dc:creator" content="Mark Birbeck">
</div>

and really, to make the markup consistent, you may as well not use the src of the image at all in the RDFa either:

<div about="photo1.jpg" typeof="http://xmlns.com/foaf/0.1/Image">
  <span rel="license" href="http://creativecommons.org/licenses/by/2.0/"></span>
  <span property="http://purl.org/dc/terms/creator" content="Mark Birbeck"></span>
  <span property="http://purl.org/dc/terms/title" content="Picture of Mark"></span>
  <time property="http://purl.org/dc/terms/created" content="2009-03-17" datatype="xsd:date" datetime="2009-03-17"></time>
  <img src="photo1.jpg">
</div>

So it’s not clear to me that interpreting the src attribute as the subject of triples offers such a huge advantage that it’s worth the inconvenience that it brings for the simple things, such as having to use:

<span rel="image"><img src="google-logo.png" alt="Google"></span>

rather than:

<img property="image" src="google-logo.png" alt="Google">

Link relations

This isn’t so much a clash between RDFa and microdata as between the interpretation that RDFa has for the rel attribute and that specified in HTML.

The built-in rel values in HTML are a bit of a mix. Some of them, like alternate, prev and next encode relationships between the document in which the link appears and another document. Others, such as bookmark and help, create relationships between the context in which the link is found and the referenced document. Still others, like nofollow, noreferrer and prefetch, are really instructions to the client about how to manage the act of traversing the link.

It doesn’t seem semantically correct to automatically create relationships based on the built-in HTML rel values, unless you are deliberately trying to extract document semantics from the page. This is a problem for RDFa, which reuses the rel attribute to provide property values for the embedded data.

One thing that could be done would be for RDFa to consistently use the property attribute everywhere rather than the rel attribute. This would not only ease the overloading but also reduce the confusion for users, who currently have to work out which attribute to use based on whether the value is a resource or a literal.

Possible Subset of RDFa

When mapping from microdata to RDFa, the only attributes that are really needed are:

vocab to define a vocabulary for the types and properties within its scope (not technically necessary, but keeps the markup simple compared to spelling out URIs for everything)
typeof to define the type of a resource or indicate a new blank node
about to provide a URI for a resource or a local identifier for a blank node
property and rel to define property names (though see above for discussion about dropping rel)
href, src and content to provide values (and datetime assuming that is supported)

In the mappings in the analysis below, I did also use the resource attribute, but only to create a reference to a blank node that was described elsewhere, when replicating the functionality of itemref. If RDFa were to enable <link> and <meta> in content in the same way as microdata, resource functionality could be replicated using <link>; as it is, you can get away with using an empty <a> element.

Similarly, I only used datatype when providing a datatype for date/time values, something that could be done automatically by RDFa. But this isn’t surprising given that microdata doesn’t support datatypes at all and the examples I was using for the mapping were from the microdata specification.

There was no need for:

prefix which defines prefixes to simplify references to properties and classes; this is hardly surprising as few of the microdata examples involved mixing namespaces, but it’s notable that the built-in prefixes of rdf: and xsd: were useful
profile which is a pointer to an external document that defines a set of terms; this is being dropped from RDFa in any case

I also kept to a simplified version of the syntax in which each property element only provided one value. This subset is basically:

resource elements can have about (equivalent to itemid) and typeof (equivalent to itemtype) attributes on them
property elements can have property or rel (equivalent to itemprop), and a value-providing attribute on them such as href or content
no element is both a resource element and a property element; to provide a property whose value is a resource, nest the resource element within the property element (using “hanging rel” processing)
no property element should provide more than one value for a property; in particular, a “hanging rel” should only have a single resource element child

This simplified profile of RDFa is fairly easy to remember and maps easily to and from microdata: most attributes can be simply renamed; the only attribute that needs to be moved as well as renamed is the “hanging rel”, which moves onto the resource element and is renamed to itemprop. Note that it also means avoiding using the src attribute to encode embedded data.

In addition to sticking to this subset of attributes, developers might be advised that using HTML link relations may lead to clashes with browser or search engine interpretation of the links in the page.

Possible Subset of Microdata

Microdata is pretty minimalistic already. The only feature that developers need to be warned about is itemref, which has no RDFa equivalent at the moment.

Guidelines for Vocabulary Authors

There are a several guidelines that come out of this comparison for people putting together vocabularies that aim to be usable in both RDFa and microdata:

The classes in the vocabulary should be distinct, or subclasses created with any relevant combinations of superclasses, so that publishers don’t have to assign more than one type to an item/resource. This restriction helps with using the vocabulary with microdata, which assumes that every item has a single type.
Provide explicit classes for everything which you anticipate might be given an identifier, as microdata doesn’t (currently) enable items to have an identifier without also having a type.
Put classes and properties in the same namespace, but do not name classes and properties with the same local name; while this doesn’t matter in microdata because the properties are interpreted relative to the class, standard conversions to RDF will create a class and a property with the same URI. URIs are case-sensitive to a simple way of ensuring that there aren’t clashes is to follow the usual RDF convention of beginning class names with an upper-case letter and property names with a lower-case letter.
Avoid property names that contain dots, as these aren’t allowed in non-URI property names in microdata.
Ensure that properties either only expect one type of value or expect values whose type can be sniffed based on the syntax of the value. If publishers use microdata, they will not be able to indicate the type of a value through the markup.
Be aware that consumers of microdata using your vocabulary will have to use the DOM API to identify the language used in any strings, and that language information won’t be carried through the standard microdata JSON serialisation (used by drag-and-drop, for example). If you anticipate multi-lingual use of your vocabulary, you may way to define a MultiLingual class with value and language properties that people can use as nested items. (It may be useful for this class and properties to be defined in the proposed ‘global’ W3C namespace so that it can be used anywhere.) If you know what languages will be used then provide separate properties for each language (eg for UK legislation I know the languages are English and Welsh so on a vocabulary for UK Legislation I could have title-en and title-cy properties).
To make markup cleaner, only reuse properties from other vocabularies on your classes if they have built-in prefixes (eg unless rdfs: is built-in to microdata as well as RDFa, don’t use rdfs:label to provide a label, but create your own label property). On the other hand, do reuse classes from other vocabularies if you don’t need to add any specialised properties to them. Note that avoiding reuse has the unfortunate side-effect of not enabling processors that understand these other vocabularies to process your data.
Avoid having properties whose values need to be retrieved in order, as these are hard to represent in RDFa. Instead, use properties with distinct names when position is important. (Yes, I know this sucks.)

Choosing Between Microdata and RDFa

The choices developers make between microdata and RDFa will, I suspect, be largely dictated by what their consumers/toolsets/publishers will support. Nevertheless, there are some features that are better supported by one or other format and might therefore sway developers one way or another:

multi-lingual embedded data is better supported in RDF than microdata+JSON
explicit datatypes for values can be provided by RDFa but not microdata
resources with multiple types are a lot easier to describe in RDFa
property values that include markup are a lot easier to write in RDFa
mixed vocabulary use is a bit easier in RDFa than in microdata
HTML5 link relations may be misinterpreted by RDFa processors
properties with list values are much easier to support in microdata
common content adopted by multiple entities is much easier in microdata

Final Words

I have no doubt that developers would be better off if there were only one recommended way of embedding data in HTML (so long as it met their requirements of course). But realistically that is, and always has been, a long shot, given the entrenched positions of the microdata and RDFa communities.

Regardless, there are lessons that RDFa and microdata could learn from each other, and changes to both languages that would help developers use them on their own, switch between them and mix them in the same document. I expect and welcome debate about the viability and effectiveness of the changes and guidelines that I’ve suggested here.

Investigating those lessons, documenting those changes and generating those guidelines was something that I had hoped the microdata/RDFa task force would be able to do. The other question to ask, given the argument that there shouldn’t be a task force at all if it’s not going to be able to bring the languages together, is whether this kind of analysis is worthwhile, and worth publishing as something more official than a blog post?

Mapping RDFa to Microdata

Jeni — Sat, 20 Aug 2011 16:38:38 +0000

This post is part of a three-part series that analyses the differences in features and syntax between microdata and RDFa. The series attempts:

to identify the differences in approach and functionality of the two languages, which should help developers choose between them
to identify any guidelines for developers of vocabularies for use with both languages
to identify a subset of functionality that is common between the two languages, which developers might want to stick to to make switching and mixing easier
to identify mapping rules that might be applied to automatically or manually map from one language to another if the simple subset is used

I’ve done this by looking at converting microdata examples to RDFa and vice versa, and the lessons to be drawn from that exercise. The three posts are on:

This post is the second of these, which looks at how RDFa might be mapped to microdata. In this case, I will aim to express the RDF created from the RDFa as the equivalent microdata JSON, and aim to create that JSON with the microdata.

To create the microdata JSON, I’ve used the rule that the URI of the first type of a resource is processed to provide a namespace that is stripped from the URIs of the properties (to create simple names where possible). In addition, when a resource has no properties, it will be represented as a string (URI) value rather than as a nested item. Other than that I hope the mapping will be obvious; I’ll point out where it involves a loss of information. I’m assuming that the document is at http://example.org/ throughout.

I have based what’s written here on the latest specifications of both microdata (in its WHAT WG and W3C variants) and RDFa Core and HTML+RDFa but I haven’t consulted with anyone involved in these efforts and may well have got things wrong. Philip Jägenstedt’s Live Microdata service has proved invaluable for testing, so many thanks to him for providing that service.

The post is rather heavy going and you might want to just skip to the summary instead of reading the whole thing.

The post goes through the examples from the RDFa specification plus one additional example from the wild. I haven’t included examples that don’t illustrate anything new, so there are some that are skipped. Other examples would be welcome.

Page Metadata

When parsing begins, the current subject will be the IRI of the document being parsed, or a value as set by a Host Language-provided mechanism (e.g., the base element in (X)HTML). This means that by default any metadata found in the document will concern the document itself:
<html>
  <head>
    <title>Jo's Friends and Family Blog</title>
    <link rel="foaf:primaryTopic" href="#bbq" />
    <meta property="dc:creator" content="Jo" />
  </head>
  <body>
    ...
  </body>
</html>

The equivalent microdata JSON for this document that we’d want to create is (note: invalid example):

{ "items": [
  {
    "id": "http://example.org/" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/primaryTopic": [ "http://example.org/#bbq" ],
      "http://purl.org/dc/terms/creator": [ "Jo" ]
    }
  }
]}

and we’d want to create it with (note: invalid example):

<html>
  <head itemscope itemid=".">
    <title>Jo's Friends and Family Blog</title>
    <link itemprop="http://xmlns.com/foaf/0.1/primaryTopic" href="#bbq" />
    <meta itemprop="http://purl.org/dc/terms/creator" content="Jo" />
  </head>
  <body>
    ...
  </body>
</html>

However, this is is not valid according to the microdata specification. In microdata, only items that have types are allowed to have identifiers. Rather than losing the identifier, we’ll add a type; I’m going to use rdfs:Resource. It’s not the nicest of URIs to type, but it’s got something close to the correct semantics. So we’ll aim for the microdata JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://example.org/" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/primaryTopic": [ "http://example.org/#bbq" ] ,
      "http://purl.org/dc/terms/creator": [ "Jo" ]
    }
  }
]}

which means we need:

<html>
  <head itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid=".">
    <title>Jo's Friends and Family Blog</title>
    <link itemprop="http://xmlns.com/foaf/0.1/primaryTopic" href="#bbq" />
    <meta itemprop="http://purl.org/dc/terms/creator" content="Jo" />
  </head>
  <body>
    ...
  </body>
</html>

Notes:

The itemscope is necessary for the page to be recognised as containing any data at all.
The itemid can’t just be empty: the . is the shortest URI you can use to reference the page itself.
I put the itemscope, itemtype and itemid on the <head> element rather than the <html> element so that they wouldn’t be inherited into the <body>: it seems to make sense for any data within the <head> to be about the page itself.
The foaf: and dc: prefixes are built-in to RDFa, so it’s easy for people to use classes and properties in those common vocabularies without having to remember their full URI. In microdata, that URI and the one for the rdfs:Resource class have to be spelled out in full.

Base URI

In (X)HTML the value of base may change the initial value of current subject:

<html>
  <head>
    <base href="http://www.example.org/jo/blog" />
    <title>Jo's Friends and Family Blog</title>
    <link rel="foaf:primaryTopic" href="#bbq" />
    <meta property="dc:creator" content="Jo" />
  </head>
  <body>
    ...
  </body>
</html>

This changes the id of the item generated:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://www.example.org/jo/blog" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/primaryTopic": [ "http://www.example.org/jo/blog#bbq" ] ,
      "http://purl.org/dc/terms/creator": [ "Jo" ]
    }
  }
]}

In the microdata, the itemid can still be . as the base URI set by the <base> element is used to resolve it:

<html>
  <head itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid=".">
    <base href="http://www.example.org/jo/blog" />
    <title>Jo's Friends and Family Blog</title>
    <link itemprop="http://xmlns.com/foaf/0.1/primaryTopic" href="#bbq" />
    <meta itemprop="http://purl.org/dc/terms/creator" content="Jo" />
  </head>
  <body>
    ...
  </body>
</html>

Explicit Subjects / ItemIds

To illustrate how this affects the statements, note in this markup how the properties inside the (X)HTML body element become part of a new calendar event object, rather than referring to the document as they do in the head of the document:

<html prefix="cal: http://www.w3.org/2002/12/cal/ical#">
  <head>
    <title>Jo's Friends and Family Blog</title>
    <link rel="foaf:primaryTopic" href="#bbq" />
    <meta property="dc:creator" content="Jo" />
  </head>
  <body>
    <p about="#bbq" typeof="cal:Vevent">
      I'm holding
      <span property="cal:summary">
        one last summer barbecue
      </span>,
      on
      <span property="cal:dtstart" content="2015-09-16T16:00:00-05:00" 
            datatype="xsd:dateTime">
        September 16th at 4pm
      </span>.
    </p>
  </body>
</html>

In microdata JSON, this would be:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://example.org/" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/primaryTopic": [ "http://example.org/#bbq" ],
      "http://purl.org/dc/terms/creator": [ "Jo" ]
    }
  } ,
  {
    "type": "http://www.w3.org/2002/12/cal/ical#Vevent" ,
    "id": "http://example.org/#bbq" ,
    "properties": {
      "summary": [ "one last summer barbecue" ] ,
      "dtstart": [ "2015-09-16T16:00:00-05:00" ] ,
    }
  }
]}

Note that this mapping loses the fact that the value of the dtstart property is a date-and-time. Processors of this JSON are expected to know that the dtstart property takes a date/time value and would have to sniff the value to work out that it’s a date-and-time rather than a date.

In-browser microdata processors can identify the value as a date/time value because the property element itself is accessed through the element.properties IDL attribute; processors that work with this DOM API can tell that it’s a <time> element, get hold of the date/time itself and access the content of the element for the human-readable representation used on the page. However, this information isn’t part of the core microdata data model.

To create this JSON from microdata you need:

<html>
  <head itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid=".">
    <title>Jo's Friends and Family Blog</title>
    <link itemprop="http://xmlns.com/foaf/0.1/primaryTopic" href="#bbq" />
    <meta itemprop="http://purl.org/dc/terms/creator" content="Jo" />
  </head>
  <body>
    <p itemscope itemid="#bbq" itemtype="http://www.w3.org/2002/12/cal/ical#Vevent">
      I'm holding
      <span itemprop="summary">
        one last summer barbecue
      </span>,
      on
      <time itemprop="dtstart" datetime="2015-09-16T16:00:00-05:00">
        September 16th at 4pm
      </time>.
    </p>
  </body>
</html>

Notes:

There are no prefix definitions in microdata, so the type has to be spelled out in full. However, with the mapping I’m assuming from RDFa to microdata JSON, the properties in that same namespace for items in that class don’t.
The itemscope has to be added despite the <p> element having both an itemid and an itemtype; if the itemscope is forgotten, the item isn’t recognised.
The original <span> element has to be changed to a <time> element because it isn’t conformant microdata for a date/time value to be supplied by any other element.

Items from the `src` Attribute

If @about is not present, then @src is next in priority order, for setting the subject of a statement. A typical use would be to indicate the licensing type of an image:
<img src="photo1.jpg" rel="license" 
     resource="http://creativecommons.org/licenses/by/2.0/" />

This should generate the microdata JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://example.org/photo1.jpg" ,
    "properties": {
      "http://www.w3.org/1999/xhtml/vocab#license": [ "http://creativecommons.org/licenses/by/2.0/" ] ,
    }
  }
]}

The src attribute in microdata is only used for a value, so creating the microdata about the image means a wrapper <span> element and a separate <link> element:

 <span itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid="photo1.jpg">
   <img src="photo1.jpg" />
   <link itemprop="http://www.w3.org/1999/xhtml/vocab#license" href="http://creativecommons.org/licenses/by/2.0/" />
 </span>

Note:

The license property is part of the built-in set of link relationships in HTML, but there is no easy way to refer to that property from microdata; they have to be spelled out as full URLs.

Additional Properties for Images

Since there is no difference between @src and @about, then the information expressed in the last example in the section on @about (the creator of an image), could be expressed as follows:
<img src="photo1.jpg"
  rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
  property="dc:creator" content="Mark Birbeck"
/>

This is a simple additional property in the microdata JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://example.org/photo1.jpg" ,
    "properties": {
      "http://www.w3.org/1999/xhtml/vocab#license": [ "http://creativecommons.org/licenses/by/2.0/" ] ,
      "http://purl.org/dc/terms/creator": [ "Mark Birbeck" ]
    }
  }
]}

which can be created through a <meta> element within the <span>:

<span itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid="photo1.jpg">
  <img src="photo1.jpg" />
  <link itemprop="http://www.w3.org/1999/xhtml/vocab#license" href="http://creativecommons.org/licenses/by/2.0/" />
  <meta itemprop="http://purl.org/dc/terms/creator" content="Mark Birbeck" />
</span>

Nested Images

Since normal chaining rules will apply, the image IRI can also be used to complete hanging triples:
<div about="http://www.blogger.com/profile/1109404" rel="foaf:img">
  <img src="photo1.jpg"
    rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
    property="dc:creator" content="Mark Birbeck"
  />
</div>

This should generate the microdata JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://www.blogger.com/profile/1109404" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/img": [{
        "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
        "id": "http://example.org/photo1.jpg" ,
        "properties": {
          "http://www.w3.org/1999/xhtml/vocab#license": [ "http://creativecommons.org/licenses/by/2.0/" ] ,
          "http://purl.org/dc/terms/creator": [ "Mark Birbeck" ]
        }
      }]
    }
  }
]}

The microdata to generate this is:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" 
     itemid="http://www.blogger.com/profile/1109404">
  <span itemprop="http://xmlns.com/foaf/0.1/img" 
        itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" 
        itemid="photo1.jpg">
    <img src="photo1.jpg" />
    <link itemprop="http://www.w3.org/1999/xhtml/vocab#license" href="http://creativecommons.org/licenses/by/2.0/" />
    <meta itemprop="http://purl.org/dc/terms/creator" content="Mark Birbeck" />
  </span>
</div>

Notes:

The big gotcha in this conversion is that in microdata, the foaf:img property has to be moved onto the item that is a value of that property; there’s no equivalent to the “hanging rel” processing in RDFa. A disadvantage of this is that anyone copying-and-pasting the <span> element to embed the same information about the image within their own page will have the itemprop attribute carried along with the image, into a context where the foaf:img property might not be relevant.

Types with Blank Nodes

For example, an author may wish to create markup for a person using the FOAF vocabulary, but without having a clear identifier for the item:
<div typeof="foaf:Person">
  <span property="foaf:name">Albert Einstein</span>
  <span property="foaf:givenName">Albert</span>
</div>

Now we have an explicit type, we can create microdata JSON that uses short names:

{ "items": [
  {
    "type": "http://xmlns.com/foaf/0.1/Person" ,
    "properties": {
      "name": [ "Albert Einstein" ] ,
      "givenName": [ "Albert" ] ,
    }
  }
]}

This can be generated with the microdata:

<div itemscope itemtype="http://xmlns.com/foaf/0.1/Person">
  <span itemprop="name">Albert Einstein</span>
  <span itemprop="givenName">Albert</span>
</div>

which is nice and simple.

Inherited Subject

The most usual way that an inherited subject might get set would be when the parent statement has an object that is a resource. Returning to the earlier example, in which the long name for the German_Empire was added, the following markup was used:
<div about="http://dbpedia.org/resource/Albert_Einstein">
  <span property="foaf:name">Albert Einstein</span>
  <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  <div rel="dbp:birthPlace" resource="http://dbpedia.org/resource/German_Empire" />
  <span about="http://dbpedia.org/resource/German_Empire"
    property="dbp:conventionalLongName">the German Empire</span>
</div>

The equivalent microdata JSON for this would be:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Albert_Einstein" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/name": [ "Albert Einstein" ] ,
      "http://dbpedia.org/property/dateOfBirth": [ "1879-03-14" ] ,
      "http://dbpedia.org/property/birthPlace": [ "http://dbpedia.org/resource/German_Empire" ]
    }
  } , {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/German_Empire" ,
    "properties": {
      "http://dbpedia.org/property/conventionalLongName": [ "the German Empire" ] ,
    }
  }
]}

Note that this microdata JSON could only be generated syntactically from the RDFa, not via RDF, because going via RDF would make it impossible to know whether to give the dbp:birthPlace property a string (which is a URI) value or a nested item. We’ll see the alternative version of the microdata RDF in the next example.

To create this microdata JSON, we need:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" 
     itemid="http://dbpedia.org/resource/Albert_Einstein">
  <span itemprop="http://xmlns.com/foaf/0.1/name">Albert Einstein</span>
  <time itemprop="http://dbpedia.org/property/dateOfBirth">1879-03-14</time>
  <link itemprop="http://dbpedia.org/property/birthPlace" href="http://dbpedia.org/resource/German_Empire" />
  <span itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"  
        itemid="http://dbpedia.org/resource/German_Empire">
    <span itemprop="http://dbpedia.org/property/conventionalLongName">the German Empire</span>
  </span>
</div>

Notes:

I’ve had to change two elements here: the <span> for the date of birth has become a <time> element as the value of the property is a date, and the <div> for the birth place has become a <link> element because the value of that property is a URL.
I’ve also had to add a nested <span> element as it’s not possible in microdata to have a single element describe both an item and a property for that item as it is in RDFa.

In an earlier illustration the subject and object for the German Empire were connected by removing the @resource, relying on the @about to set the object:
<div about="http://dbpedia.org/resource/Albert_Einstein">
  <span property="foaf:name">Albert Einstein</span>
  <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  <div rel="dbp:birthPlace">
    <span about="http://dbpedia.org/resource/German_Empire"
      property="dbp:conventionalLongName">the German Empire</span>
  </div>
</div>

While this generates the same RDF as the previous example, the microdata JSON that it generates should probably be different: this time, the item for the German Empire is nested within the item for Albert Einstein:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Albert_Einstein" ,
    "properties": {
      "http://xmlns.com/foaf/0.1/name": [ "Albert Einstein" ] ,
      "http://dbpedia.org/property/dateOfBirth": [ "1879-03-14" ] ,
      "http://dbpedia.org/property/birthPlace": [{
        "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
        "id": "http://dbpedia.org/resource/German_Empire" ,
        "properties": {
          "http://dbpedia.org/property/conventionalLongName": [ "the German Empire" ]
        }
      }
    }
  }
]}

To create this, the microdata needs to look like:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" 
     itemid="http://dbpedia.org/resource/Albert_Einstein">
  <span itemprop="http://xmlns.com/foaf/0.1/name">Albert Einstein</span>
  <time itemprop="http://dbpedia.org/property/dateOfBirth">1879-03-14</time>
  <div itemprop="http://dbpedia.org/property/birthPlace" 
       itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"  
       itemid="http://dbpedia.org/resource/German_Empire">
    <span itemprop="http://dbpedia.org/property/conventionalLongName">the German Empire</span>
  </div>
</div>

Note that while this looks quite similar to the RDFa version, in fact the itemid attribute that holds the URI for the German Empire is on a different element from the about attribute in the RDFa.

The third RDFa example around this same content is:

but it is also possible for authors to achieve the same effect by removing the @about and leaving the @resource:

<div about="http://dbpedia.org/resource/Albert_Einstein">
  <span property="foaf:name">Albert Einstein</span>
  <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  <div rel="dbp:birthPlace" resource="http://dbpedia.org/resource/German_Empire">
    <span property="dbp:conventionalLongName">the German Empire</span>
  </div>
</div>

This should lead to the same microdata JSON, so I won’t bother repeating the microdata. What’s interesting is that this pattern: the wrapper element containing the property (rel) and identifier for the item that is the value for that property (resource) is a lot closer to the microdata pattern of expressing nested items. The big distinction here is that while in microdata, the itemtype also resides on that element, if you tried adding a typeof attribute to the inner <div> in RDFa, you’d end up with a new blank node.

Anonymous Nested Resources

However, an author could just as easily say that Spinoza influenced something by the name of Albert Einstein, that was born on March 14th, 1879:
<div about="http://dbpedia.org/resource/Baruch_Spinoza" rel="dbp-owl:influenced">
  <div>
    <span property="foaf:name">Albert Einstein</span>
    <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  </div>
</div>

This should generate the microdata JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Baruch_Spinoza" ,
    "properties": {
      "http://dbpedia.org/ontology/influenced": [{
        "properties": {
          "http://xmlns.com/foaf/0.1/name": [ "Albert Einstein" ] ,
          "http://dbpedia.org/property/dateOfBirth": [ "1879-03-14" ]
        }
      }]
    }
  }
]}

which again means moving an attribute in microdata:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"
     itemid="http://dbpedia.org/resource/Baruch_Spinoza">
  <div itemprop="http://dbpedia.org/ontology/influenced"
       itemscope>
    <span itemprop="http://xmlns.com/foaf/0.1/name">Albert Einstein</span>
    <time itemprop="http://dbpedia.org/property/dateOfBirth">1879-03-14</time>
  </div>
</div>

It is generally harder to move to microdata from RDFa when the RDFa has an element that both provides a subject and provides a property.

The RDFa spec provides a couple of additional methods of marking up the same content to give exactly the same RDF (and microdata JSON):

Note that the div is superfluous, and an RDFa Processor will create the intermediate object even if the element is removed:
<div about="http://dbpedia.org/resource/Baruch_Spinoza" rel="dbp-owl:influenced">
  <span property="foaf:name">Albert Einstein</span>
  <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
</div>
An alternative pattern is to keep the div and move the @rel onto it:
<div about="http://dbpedia.org/resource/Baruch_Spinoza">
  <div rel="dbp-owl:influenced">
    <span property="foaf:name">Albert Einstein</span>
    <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  </div>
</div>
From the point of view of the markup, this latter layout is to be preferred, since it draws attention to the ‘hanging rel’. But from the point of view of an RDFa Processor, all of these permutations need to be supported.

Interestingly, it’s this latter permutation that is the one that’s closest to the microdata method of expressing the data, though as we will see in the next section, the “hanging rel” is not exactly equivalent to the itemprop on the wrapper element.

Hanging Rels

Note that each occurrence of @about will complete any incomplete triples. For example, to mark up the fact that Albert Einstein had a residence both in the German Empire and Switzerland, an author need only specify one @rel value that is then used with multiple @about values:
<div about="http://dbpedia.org/resource/Albert_Einstein" rel="dbp-owl:residence">
  <span about="http://dbpedia.org/resource/German_Empire" />
  <span about="http://dbpedia.org/resource/Switzerland" />
</div>

The data embedded here gives two values for the dbp-owl:residence property:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Albert_Einstein" ,
    "properties": {
      "http://dbpedia.org/ontology/residence": [
        "http://dbpedia.org/resource/German_Empire" ,
        "http://dbpedia.org/resource/Switzerland"
      ]
    }
  }
]}

In microdata, the itemprop attribute has to appear on both the nested elements to make it clear that they both provide values for that property:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" 
     itemid="http://dbpedia.org/resource/Albert_Einstein">
 <link itemprop="http://dbpedia.org/ontology/residence" 
       href="http://dbpedia.org/resource/German_Empire" />
 <link itemprop="http://dbpedia.org/ontology/residence"
       href="http://dbpedia.org/resource/Switzerland" />
</div>

The next example illustrates this with nested items rather than strings:

To illustrate, to indicate that Spinoza influenced both Einstein and Schopenhauer, the following markup could be used:

<div about="http://dbpedia.org/resource/Baruch_Spinoza">
  <div rel="dbp-owl:influenced">
    <div typeof="foaf:Person">
      <span property="foaf:name">Albert Einstein</span>
      <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
    </div>
    <div typeof="foaf:Person">
      <span property="foaf:name">Arthur Schopenhauer</span>
      <span property="dbp:dateOfBirth" datatype="xsd:date">1788-02-22</span>
    </div>          
  </div>
</div>

This should generate:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Baruch_Spinoza" ,
    "properties": {
      "http://dbpedia.org/ontology/influenced": [{
        "type": "http://xmlns.com/foaf/0.1/Person" ,
        "properties": {
          "name": [ "Albert Einstein" ] ,
          "http://dbpedia.org/property/dateOfBirth": [ "1879-03-14" ]
        }
      }, {
        "type": "http://xmlns.com/foaf/0.1/Person" ,
        "properties": {
          "name": [ "Arthur Schopenhauer" ] ,
          "http://dbpedia.org/property/dateOfBirth": [ "1788-02-22" ]
        }
      }]
    }
  }
]}

In this case, the itemprop that is equivalent to the RDFa rel has to move down onto the elements representing the items:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"
     itemid="http://dbpedia.org/resource/Baruch_Spinoza">
  <div>
    <div itemprop="http://dbpedia.org/ontology/influenced"
         itemscope itemtype="http://xmlns.com/foaf/0.1/Person">
      <span itemprop="name">Albert Einstein</span>
      <time itemprop="http://dbpedia.org/property/dateOfBirth">1879-03-14</time>
    </div>
    <div itemprop="http://dbpedia.org/ontology/influenced"
         itemscope itemtype="http://xmlns.com/foaf/0.1/Person">
      <span itemprop="name">Arthur Schopenhauer</span>
      <time itemprop="http://dbpedia.org/property/dateOfBirth">1788-02-22</time>
    </div>
  </div>
</div>

The wrapper <div> around both items isn’t necessary; I’ve left it to stay as close to the markup of the original RDFa as possible.

Implicit Resources

Triples are also ‘completed’ if any one of @property, @rel or @rev are present. However, unlike the situation when @about or @typeof are present, all predicates are attached to one bnode:
<div about="http://dbpedia.org/resource/Baruch_Spinoza" rel="dbp-owl:influenced">
  <span property="foaf:name">Albert Einstein</span>
  <span property="dbp:dateOfBirth" datatype="xsd:date">1879-03-14</span>
  <div rel="dbp-owl:residence">
    <span about="http://dbpedia.org/resource/German_Empire" />
    <span about="http://dbpedia.org/resource/Switzerland" />
  </div>
</div>

To be equivalent to the RDF generated from this markup, the microdata JSON would be:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://dbpedia.org/resource/Baruch_Spinoza" ,
    "properties": {
      "http://dbpedia.org/ontology/influenced": [{
        "properties": {
          "http://xmlns.com/foaf/0.1/name": [ "Albert Einstein" ] ,
          "http://dbpedia.org/property/dateOfBirth": [ "1879-03-14" ]
          "http://dbpedia.org/ontology/residence": [
            "http://dbpedia.org/resource/German_Empire" ,
            "http://dbpedia.org/resource/Switzerland"
          ]
        }
      }]
    }
  }
]}

Microdata is a lot more explicit about when items get created, and consequently requires a bit more markup:

<div itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"
     itemid="http://dbpedia.org/resource/Baruch_Spinoza">
  <div itemprop="http://dbpedia.org/ontology/influenced" itemscope>
    <span itemprop="name">Albert Einstein</span>
    <time itemprop="http://dbpedia.org/property/dateOfBirth">1879-03-14</time>
    <div>
     <link itemprop="http://dbpedia.org/ontology/residence" 
           href="http://dbpedia.org/resource/German_Empire" />
     <link itemprop="http://dbpedia.org/ontology/residence"
           href="http://dbpedia.org/resource/Switzerland" />
    </div>
  </div>
</div>

Overriding Text Content

The value of @content is given precedence over any element content, so the following would give exactly the same triple as shown above:
<span about="http://internet-apps.blogspot.com/"
      property="dc:creator" content="Mark Birbeck">John Doe</span>

The equivalent microdata should generate the JSON:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://internet-apps.blogspot.com/" ,
    "properties": {
      "http://purl.org/dc/terms/creator": [ "Mark Birbeck" ]
    }
  }
]}

Only the <time> element and links override the content of an element in microdata. So a mirror of this example needs a separate element:

  <span itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"
        itemid="http://internet-apps.blogspot.com/">
    <meta itemprop="http://purl.org/dc/terms/creator" content="Mark Birbeck" />
    John Doe
  </span>

Language Support

In RDFa the Host Language may provide a mechanism for setting the language tag. In XHTML+RDFa [XHTML-RDFA], for example, the XML language attribute @xml:lang or the attribute @lang is used to add this information, whether the plain literal is designated by @content, or by the inline text of the element:
<meta about="http://example.org/node"
  property="ex:property" xml:lang="fr" content="chat" />

Like the datatype of a value, the language of a value isn’t captured by the microdata data model or the JSON representation of that data model. So the fact that ‘chat’ is French is lost:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://example.org/node" ,
    "properties": {
      "http://example.org/property": [ "chat" ]
    }
  }
]}

The equivalent microdata is thus:

<span itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource"
      itemid="http://example.org/node"
  <meta itemprop="ex:property" xml:lang="fr" content="chat" />
</span>

with the language only accessible if you are using the DOM to process the microdata.

Literals that Include Markup

RDFa therefore supports the use of normal markup to express XML literals, by using @datatype:
<h2 property="dc:title" datatype="rdf:XMLLiteral">
  E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
</h2>

The datatype="rdf:XMLLiteral" acts like a flag to indicate that the serialised content of the element (innerHTML) needs to be used as the value of the property, rather than the textContent, which includes markup, can be expressed in microdata JSON as follows:

{ "http://purl.org/dc/terms/title": "E = mc<sup>2</sup>: The Most Urgent Problem of Our Time" }

There’s no way to generate this in microdata except by repeating the escaped version of the content in a content attribute:

<h2>
  E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
  <meta itemprop="http://purl.org/dc/terms/title"
    content="E = mc&lt;sup>2&lt;/sup>: The Most Urgent Problem of Our Time" />
</h2>

This is hardly ideal. It’s tedious enough with a short string like this one; for larger amounts of information such as long descriptions of an event, it would be very tedious.

The `resource` Attribute

RDFa provides the @resource attribute as a way to set the object of statements. This is particularly useful when referring to resources that are not themselves navigable links:

<html>
  <head>
    <title>On Crime and Punishment</title>
    <base href="http://www.example.com/candp.xhtml" />
  </head>
  <body>
    <blockquote about="#q1" rel="dc:source" resource="urn:ISBN:0140449132" >
      <p id="q1">
        Rodion Romanovitch! My dear friend! If you go on in this way
        you will go mad, I am positive! Drink, pray, if only a few drops!
      </p>
    </blockquote>
  </body>
</html>

This should produce:

{ "items": [
  {
    "type": "http://www.w3.org/2000/01/rdf-schema#Resource" ,
    "id": "http://www.example.com/candp.xhtml#q1" ,
    "properties": {
      "http://purl.org/dc/terms/source": [ "urn:ISBN:0140449132" ]
    }
  }
]}

which is expressed through:

<html>
  <head>
    <title>On Crime and Punishment</title>
    <base href="http://www.example.com/candp.xhtml" />
  </head>
  <body>
    <blockquote itemscope itemtype="http://www.w3.org/2000/01/rdf-schema#Resource" itemid="#q1">
      <link itemprop="http://purl.org/dc/terms/source" href="urn:ISBN:0140449132" />
      <p id="q1">
        Rodion Romanovitch! My dear friend! If you go on in this way
        you will go mad, I am positive! Drink, pray, if only a few drops!
      </p>
    </blockquote>
  </body>
</html>

The property and value have to be moved onto a nested <link>, but this is a more extensible pattern than the RDFa method as it enables other properties to be expressed in the same way.

Multiple Types

This last example comes from the wild rather than being an example in the specification. At http://bitmunk.com/browse we find:

<span about="http://bitmunk.com/about#service" 
      typeof="vcard:VCard commerce:Business gr:BusinessEntity" 
      property="rdfs:label vcard:fn">Bitmunk</span>

This shows the use of multiple types and of multiple properties with the same value, because the pages are attempting to use multiple vocabularies that cover the same domain (organisations) to different depths. In the equivalent microdata, we have to choose one of the types; I’m going to assume that it should just use the first one from the typeof attribute:

{ "items": [
  {
    "type": "http://www.w3.org/2006/vcard/ns#VCard" ,
    "id": "http://bitmunk.com/about#service" ,
    "properties": {
      "http://www.w3.org/1999/02/22-rdf-syntax-ns#type": [
        "http://purl.org/commerce#Business" ,
        "http://purl.org/goodrelations/v1#BusinessEntity"
      ] ,
      "http://www.w3.org/2000/01/rdf-schema#label": [ "Bitmunk" ] ,
      "fn": [ "Bitmunk" ]
    }
  }
]}

The microdata equivalent is:

<span itemscope itemid="http://bitmunk.com/about#service" 
      itemtype="http://www.w3.org/2006/vcard/ns#VCard">
  <link itemprop="http://www.w3.org/1999/02/22-rdf-syntax-ns#type"
       href="http://purl.org/commerce#Business" />
  <link itemprop="http://www.w3.org/1999/02/22-rdf-syntax-ns#type"
       href="http://purl.org/goodrelations/v1#BusinessEntity" />
  <span itemprop="http://www.w3.org/2000/01/rdf-schema#label fn">Bitmunk</span>
</span>

Notes:

Technically, the RDFa doesn’t place any ordering on the three classes, but I’m picking the first for the purpose of the microdata conversion. The other classes are harder to get at in the JSON: they have to be referenced via the rdf:type microdata property rather than the type JSON property. Consumers that are on the lookout for items of the type gr:BusinessEntity wouldn’t spot these items.

Mapping Microdata to RDFa

Jeni — Sat, 20 Aug 2011 16:35:28 +0000

This post is part of a three-part series that analyses the differences in features and syntax between microdata and RDFa. The series attempts:

to identify the differences in approach and functionality of the two languages, which should help developers choose between them
to identify any guidelines for developers of vocabularies for use with both languages
to identify a subset of functionality that is common between the two languages, which developers might want to stick to to make switching and mixing easier
to identify mapping rules that might be applied to automatically or manually map from one language to another if the simple subset is used

I’ve done this by looking at converting microdata examples to RDFa and vice versa, and the lessons to be drawn from that exercise. The three posts are on:

This post is the first of these, which looks at how microdata might be mapped to RDFa, in terms of generating the same RDF according to the microdata-to-RDF mapping rules that I outlined in my post on Microdata + RDF.

I have based what’s written here on the latest specifications of both microdata (in its WHAT WG and W3C variants) and RDFa Core and HTML+RDFa but I haven’t consulted with anyone involved in these efforts and may well have got things wrong. Gregg Kellogg’s Distiller service has proved invaluable for testing, so many thanks to him for providing that service.

The post is rather heavy going and you might want to just skip to the summary instead of reading the whole thing.

The post goes through the examples from the microdata specification (most of them are in both versions, the only exceptions being those that use the vCard vocabulary). I haven’t included examples that don’t illustrate anything new, so there are some that are skipped. Other examples would be welcome.

Unidentified Items / Blank Node Subjects

Here there are two items, each of which has the property “name”:

<div itemscope>
 <p>My name is <span itemprop="name">Elizabeth</span>.</p>
</div>

<div itemscope>
 <p>My name is <span itemprop="name">Daniel</span>.</p>
</div>

The first challenge is to map this into RDFa because the properties are tokens rather than URIs and there is no type for either of the items. What I’ll assume here is that the name properties are local to the document itself and thus the equivalent RDF is:

[ <#name> "Elizabeth" ] .
[ <#name> "Daniel" ] .

This can be achieved in RDFa through either:

<div vocab="#" about="_:elizabeth">
  <p>My name is <span property="name">Elizabeth</span>.</p>
</div>

<div vocab="#" about="_:daniel">
  <p>My name is <span property="name">Daniel</span>.</p>
</div>

or:

<div vocab="#" typeof>
  <p>My name is <span property="name">Elizabeth</span>.</p>
</div>

<div vocab="#" typeof>
  <p>My name is <span property="name">Daniel</span>.</p>
</div>

Notes:

The vocab="#" sets the vocabulary to the location of the current document (plus an empty fragment identifier); this URI is then concatenated to the property token (name) to create a URI that is unique to the document. In a document such as this it would make sense to put the vocab="#" attribute on the <html> element rather than on every single item.
With no type in sight, blank nodes can either be created by having an empty typeof attribute or through an about attributes whose value starts with _:. The latter has the advantage of providing an identifier for the blank node that can be used elsewhere in the document, but the former is shorter so will be used where possible in the remaining examples of this post.

Values from the `src` Attribute

The next example introduces the use of the src attribute to set the value of the property.

In this example, the item has one property, “image”, whose value is a URL:
<div itemscope>
 <img itemprop="image" src="google-logo.png" alt="Google">
</div>

This should probably be mapped to the RDF:

[ <#image> <google-logo.png> ] .

The difficulty with this is that in RDFa, the src attribute is used for the subject of a statement (equivalent to a microdata item) rather than the object (equivalent to a microdata value). So we have two choices for equivalent RDFa. One is to use a similar pattern to that used above, but introduce a wrapper element that provides the property:

<div vocab="#" typeof>
 <span rel="image"><img src="google-logo.png" alt="Google"></span>
</div>

Another is to provide what would normally be an object through a resource attribute and then use a rev attribute (rather than the usual rel) attribute to reverse the relationship:

<div vocab="#">
 <img resource="_:thing" rev="image" src="google-logo.png" alt="Google">
</div>

This has three disadvantages over the first option:

the resource attribute that creates the item is on the <img> element rather than on the wrapper <div> which makes it hard to create other properties for that item
we have to use a rev attribute, reversing the normal flow of relationships; I (at least) find this hard to figure out when there’s not a rel attribute as well
we have to make up an id for the blank node we want to generate

I’ll note that it took me five or six failed attempts to generate the above options. If I hadn’t had the RDF Distiller to test with, I would have got it wrong. Note that at least through the RDF Distiller, to be recognised, the resource attribute has to have an (empty) value — it is not enough for it to simply be present, unlike with the typeof attribute. Note that the resource attribute has to explicitly point to a blank node to create a blank node rather than having the property be associated with the document in which this appears.

Values from the `datetime` Attribute

The next example illustrates the use of the <time> element to provide a date/time value for a property.

In this example, the item has one property, “birthday”, whose value is a date:
<div itemscope>
 I was born on <time itemprop="birthday" datetime="2009-05-10">May 10th 2009</time>.
</div>

I’m assuming this should map to the RDF:

[ <#birthday> "2009-05-10"^^<http://www.w3.org/2001/XMLSchema#date> ]

There is an open issue (ISSUE-97) about this on RDFa, which currently requires the use of the content attribute to provide the value as follows:

<div vocab="#" typeof>
 I was born on <time property="birthday" content="2009-05-10" datatype="xsd:date" datetime="2009-05-10">May 10th 2009</time>.
</div>

Note that the xsd: prefix is built-in within RDFa so there’s on need for any declaration for it, which makes it fairly easy to specify the standard date/time datatypes.

If ISSUE-97 were resolved nicely it would be possible to instead do:

<div vocab="#" typeof>
 I was born on <time property="birthday" datetime="2009-05-10">May 10th 2009</time>.
</div>

Notes:

To make this work, RDFa processors would have to look at the syntax of the datetime attribute to work out what datatype the value should be matched to.
The syntax permitted in the datetime attribute isn’t exactly the same as that permitted by the XML Schema time and dateTime types usually used in RDF (and XML), in that the seconds component is optional within HTML. The resolution to ISSUE-97 will need to take this into account. Otherwise, anyone mapping from microdata to RDFa manually will need to ensure that the content attribute includes the seconds component.

Nested Items / Object Properties

In this example, the outer item represents a person, and the inner one represents a band:
<div itemscope>
 <p>Name: <span itemprop="name">Amanda</span></p>
 <p>Band: <span itemprop="band" itemscope> <span itemprop="name">Jazz Band</span> (<span itemprop="size">12</span> players)</span></p>
</div>
The outer item here has two properties, “name” and “band”. The “name” is “Amanda”, and the “band” is an item in its own right, with two properties, “name” and “size”. The “name” of the band is “Jazz Band”, and the “size” is “12”.

The equivalent RDF for this example would be:

[
  <#name> "Amanda" ;
  <#band> [
    <#name> "Jazz Band" ;
    <#size> "12"
  ]
]

Note that the size property is just a plain literal value; unlike with date/times, there’s no way to tell from the microdata that the value is a number.

In RDFa this could be done with:

<div vocab="#" typeof>
 <p>Name: <span property="name">Amanda</span></p>
 <p>Band: <span rel="band"> <span property="name">Jazz Band</span> (<span property="size">12</span> players)</span></p>
</div>

This follows the microdata fairly closely but note that the nested resource doesn’t need an empty typeof attribute: it’s only the top-level items that do. It might be easier, for consistency and extensibility, to always include an explicit nested element (with an empty typeof attribute in this case) to represent the nested resource:

<div vocab="#" typeof>
 <p>Name: <span property="name">Amanda</span></p>
 <p>Band: <span rel="band"><span typeof> <span property="name">Jazz Band</span> (<span property="size">12</span> players)</span></span></p>
</div>

The other thing that people have to watch out for is that because the value of the band property is a resource rather than a literal, we have to use the rel attribute rather than the property attribute as we do elsewhere.

Itemref

This example is the same as the previous one, but all the properties are separated from their items:

<div itemscope id="amanda" itemref="a b"></div>
<p id="a">Name: <span itemprop="name">Amanda</span></p>
<div id="b" itemprop="band" itemscope itemref="c"></div>
<div id="c">
 <p>Band: <span itemprop="name">Jazz Band</span></p>
 <p>Size: <span itemprop="size">12</span> players</p>
</div>

This should create the same RDF as the previous example:

[
  <#name> "Amanda" ;
  <#band> [
    <#name> "Jazz Band" ;
    <#size> "12"
  ]
]

changing the markup as little as possible. The RDFa equivalent is:

<div id="amanda"></div>
<p vocab="#" about="_:amanda">Name: <span property="name">Amanda</span></p>
<div vocab="#" about="_:amanda" rel="band" resource="_:c"></div>
<div vocab="#" about="_:c">
 <p>Band: <span property="name">Jazz Band</span></p>
 <p>Size: <span property="size">12</span> players</p>
</div>

In microdata, the itemref attribute is a method of an item adopting name/value pairs described in a separate location within the page. In RDFa, the equivalent is to say that the name/value pairs are all related to the same resource by consistently referring to the resource as the subject of the statements. In the above case, there are two blank nodes labelled _:amanda and _:c, and the about attribute is used on the same elements that provide the properties (or a wrapper element) to indicate the identity of the subject of the statements.

Notes:

The resource attribute has to be used to indicate the blank node for the band.
As before, the rel attribute has to be used for the band property, rather than the property attribute, because the object of the statement is a resource. The rule is that if you’re using resource, you should use rel. (I used property erroneously the first time I tried to write this mapping. I will never learn.)

There is another example of itemref in use later in the microdata specification:

<!DOCTYPE HTML>
<html>
 <head>
  <title>Photo gallery</title>
 </head>
 <body>
  <h1>My photos</h1>
  <figure itemscope itemtype="http://n.whatwg.org/work" itemref="licenses">
   <img itemprop="work" src="images/house.jpeg" alt="A white house, boarded up, sits in a forest.">
   <figcaption itemprop="title">The house I found.</figcaption>
  </figure>
  <figure itemscope itemtype="http://n.whatwg.org/work" itemref="licenses">
   <img itemprop="work" src="images/mailbox.jpeg" alt="Outside the house is a mailbox. It has a leaflet inside.">
   <figcaption itemprop="title">The mailbox.</figcaption>
  </figure>
  <footer>
   <p id="licenses">All images licensed under the <a itemprop="license"
   href="http://www.opensource.org/licenses/mit-license.php">MIT
   license</a>.</p>
  </footer>
 </body>
</html>

This is equivalent to the RDF:

[
  a <http://n.whatwg.org/work> ;
  <http://n.whatwg.org/license> <http://www.opensource.org/licenses/mit-license.php> ;
  <http://n.whatwg.org/work> <images/house.jpeg> ;
  <http://n.whatwg.org/title> "The house I found." ;
] .
[
  a <http://n.whatwg.org/work> ;
  <http://n.whatwg.org/license> <http://www.opensource.org/licenses/mit-license.php> ;
  <http://n.whatwg.org/work> <images/mailbox.jpeg> ;
  <http://n.whatwg.org/title> "The mailbox." ;
] .

Note that the license property is adopted by both the items in the microdata. In this particular example, the two items have the same type, and thus the license property has the same meaning in each item. It’s also possible for itemref to be used on two items that have different types, pointing to the same element, in which case the shared properties defined within that element could mean different things for the two items.

There is no way that I am aware of within RDFa to support shared use of portions of content. There could be a rough equivalent that would work in the case where the shared properties had the same semantics if RDFa allowed the about attribute to take multiple values (note: invalid example):

<!DOCTYPE HTML>
<html>
 <head>
  <title>Photo gallery</title>
 </head>
 <body vocab="http://n.whatwg.org/">
  <h1>My photos</h1>
  <figure about="_:house" typeof="work">
   <span rel="work"><img src="images/house.jpeg" alt="A white house, boarded up, sits in a forest."></span>
   <figcaption property="title">The house I found.</figcaption>
  </figure>
  <figure about="_:mailbox" typeof="work">
   <span rel="work"><img src="images/mailbox.jpeg" alt="Outside the house is a mailbox. It has a leaflet inside."></span>
   <figcaption property="title">The mailbox.</figcaption>
  </figure>
  <footer>
   <p about="_:house _:mailbox">All images licensed under the <a rel="license"
   href="http://www.opensource.org/licenses/mit-license.php">MIT
   license</a>.</p>
  </footer>
 </body>
</html>

but this wouldn’t support the possibility of the same property having different semantics (and therefore different URIs) for the separate resources.

It’s also worth noting in this example that the mapping to RDF that I’m assuming results, in this example, in http://n.whatwg.org/work being both a class and a property. The creators of RDF vocabularies tend to name classes with an Uppercase initial letter and properties with a lowercase initial letter, and thus avoid these kinds of clashes. Vocabulary designers who are mindful of mappings to RDF may want to take the same approach.

Multiple Values

This example describes an ice cream, with two flavors:
<div itemscope>
 <p>Flavors in my favorite ice cream:</p>
 <ul>
  <li itemprop="flavor">Lemon sorbet</li>
  <li itemprop="flavor">Apricot sorbet</li>
 </ul>
</div>
This thus results in an item with two properties, both “flavor”, having the values “Lemon sorbet” and “Apricot sorbet”.

This example highlights one of the real nightmares of RDF: lists. In microdata, the order of the values ‘Lemon sorbet’ and ‘Apricot sorbet’ is naturally retained. There are three possible mappings to RDF.

Creating Multiple Statements

If the order of the flavours of ice cream in this example don’t actually matter, the equivalent RDF is:

[ <#flavor> "Lemon sorbet" , "Apricot sorbet" ]

which is equivalent to:

[ <#flavor> "Apricot sorbet" , "Lemon sorbet" ]

In this case, the RDFa is straight-forward:

<div vocab="#" typeof>
 <p>Flavors in my favorite ice cream:</p>
 <ul>
  <li property="flavor">Lemon sorbet</li>
  <li property="flavor">Apricot sorbet</li>
 </ul>
</div>

It’s surprising how common it is that order doesn’t actually matter when there are multiple values for a property, such that this mapping is quite sufficient. But I’m absolutely not going to pretend that order is never important…

Creating an `rdf:Seq`

If the order of the flavours does matter, there are two ways of representing that order using RDF. The first is to use an rdf:Seq resource. This method was the original method of representing lists in RDF and is very natural to do in RDF/XML, but has largely fallen out of favour for the second method which I’ll describe below.

Using the rdf:Seq method, the equivalent RDF for the microdata would be:

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
[
  <#flavor> [
    a rdf:Seq ;
    rdf:_1 "Lemon sorbet" ;
    rdf:_2 "Apricot sorbet"
  ]
]

which can be generated with:

<div vocab="#" typeof>
 <p>Flavors in my favorite ice cream:</p>
 <div rel="flavor">
  <ul typeof="rdf:Seq">
   <li property="rdf:_1">Lemon sorbet</li>
   <li property="rdf:_2">Apricot sorbet</li>
  </ul>
 </div>
</div>

Notes:

There are various other ways in which the namespace for the rdf:Seq could be created, but since the rdf: prefix is built-in to RDFa 1.1, it seems easier to use that than anything that explicitly writes out the full (ugly) RDF namespace.
The <div> wrapper for the <ul> is needed in the same way as the wrapper <span> element was needed in the <img> example above. Whereas in microdata, the property element also describes the value of that property, in RDFa when the object of a statement is a resource the description of that resource is nested inside the property element (in a similar way to RDF/XML).

Creating a `rdf:List`

The current recommended way to create a list in RDF is to use a rdf:List resource. This essentially uses a linked list model to represent lists, with the rdf:first item of a list being a value and the rdf:rest being either another rdf:List or rdf:nil. Spelled out, the RDF would look like:

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
[ 
  <#flavor> [
    a rdf:List ;
    rdf:first "Lemon sorbet" ;
    rdf:rest [
      rdf:first "Apricot sorbet" ;
      rdf:rest rdf:nil
    ]
  ]
]

but of course Turtle lets you write it:

[] <#flavor> ( "Lemon sorbet" "Apricot sorbet" ) .

Unfortunately, RDFa has no such syntax sugar. Which means:

<div vocab="#" typeof>
 <p>Flavors in my favorite ice cream:</p>
 <div rel="flavor">
  <ul typeof="rdf:List">
   <li property="rdf:first">Lemon sorbet</li>
   <li rel="rdf:rest">
    <span typeof="rdf:List">
     <span property="rdf:first">Apricot sorbet</span>
     <a rel="rdf:rest" href="rdf:nil"></a>
    </span>
   </li>
  </ul>
 </div>
</div>

Yep, horrific. Verbose and easy to get wrong, and that’s just for two items. If a third is added, the pattern is to add an about attribute on the middle items of the list so that the rdf:rest property which covers the next item in the list can be assigned to it. For example:

<div vocab="#" typeof>
 <p>Flavors in my favorite ice cream:</p>
 <div rel="flavor">
  <ul typeof="rdf:List">
   <li property="rdf:first">Lemon sorbet</li>
   <li rel="rdf:rest">
    <span about="_:2" typeof="List">
     <span property="rdf:first">Apricot sorbet</span>
    </span>
   </li>
   <li about="_:2" rel="rdf:rest">
     <span typeof="rdf:List">
     <span property="rdf:first">Raspberry sorbet</span>
     <a rel="rdf:rest" href="http://www.w3.org/1999/02/22-rdf-syntax-ns#nil"></a>
     </span>
   </li>
  </ul>
 </div>
</div>

Note:

I’ve used an empty <a> element with a href attribute to point to the rdf:nil resource. An alternative would be to use the resource attribute, which would have the advantage of not having to spell out the full URI for rdf:nil, but I’m trying to stick to using as few attributes as possible.
Using an empty <a> element for a link isn’t ideal; it would be neater to use a <link> element, but these aren’t allowed in flow content within HTML5 (<link> and <meta> are only permitted within the microdata specification, and then only if they have an itemprop attribute). The RDFa specification could likewise allow them.

Multiple Properties Sharing a Value

Here we see an item with two properties, “favorite-color” and “favorite-fruit”, both set to the value “orange”:
<div itemscope>
 <span itemprop="favorite-color favorite-fruit">orange</span>
</div>

This should map to the RDF:

[
  <#favorite-color> "orange" ;
  <#favorite-fruit> "orange"
]

Like itemprop, property can take multiple values, so the RDFa equivalent is simply:

<div vocab="#" typeof>
 <span property="favorite-color favorite-fruit">orange</span>
</div>

Types

Here, the item’s type is “http://example.org/animals#cat”:
<section itemscope itemtype="http://example.org/animals#cat">
 <h1 itemprop="name">Hedral</h1>
 <p itemprop="desc">Hedral is a male american domestic
 shorthair, with a fluffy black fur with white paws and belly.</p>
 <img itemprop="img" src="hedral.jpeg" alt="" title="Hedral, age 18 months">
</section>
In this example the “http://example.org/animals#cat” item has three properties, a “name” (“Hedral”), a “desc” (“Hedral is…”), and an “img” (“hedral.jpeg”).

I’ll assume that this should be mapped to the RDF:

[
  a <http://example.org/animals#cat> ;
  <http://example.org/animals#name> "Hedral" ;
  <http://example.org/animals#desc> "Hedral is a male american domestic shorthair, with a fluffy black fur with white paws and belly." ;
  <http://example.org/animals#img> <hedral.jpeg>
]

In this case, the vocab can be set to http://example.org/animals# and both the itemtype and the various property and rel attributes will use that as the basis for their identifying URIs:

<section vocab="http://example.org/animals#" typeof="cat">
 <h1 property="name">Hedral</h1>
 <p property="desc">Hedral is a male american domestic
 shorthair, with a fluffy black fur with white paws and belly.</p>
 <div rel="img"><img src="hedral.jpeg" alt="" title="Hedral, age 18 months"></div>
</section>

Global Identifiers

Here, an item is talking about a particular book:

<dl itemscope
    itemtype="http://vocab.example.net/book"
    itemid="urn:isbn:0-330-34032-8">
 <dt>Title
 <dd itemprop="title">The Reality Dysfunction
 <dt>Author
 <dd itemprop="author">Peter F. Hamilton
 <dt>Publication date
 <dd><time itemprop="pubdate" datetime="1996-01-26">26 January 1996</time>
</dl>

Here, the item has an identifier so unlike the previous examples, the subject of the statements in the RDF is no longer a blank node:

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
<urn:isbn:0-330-34032-8>
  a <http://vocab.example.net/book> ;
  <http://vocab.example.net/title> "The Reality Dysfunction\n " ;
  <http://vocab.example.net/author> "Peter F. Hamilton\n " ;
  <http://vocab.example.net/pubdate> "1996-01-26"^^xsd:date ;
  .

In RDFa, the subject is provided using the about attribute:

<dl vocab="http://vocab.example.net/"
    typeof="book"
    about="urn:isbn:0-330-34032-8">
 <dt>Title
 <dd property="title">The Reality Dysfunction
 <dt>Author
 <dd property="author">Peter F. Hamilton
 <dt>Publication date
 <dd><time property="pubdate" content="1996-01-26" datatype="xsd:date" datetime="1996-01-26">26 January 1996</time>
</dl>

Global Property Names

Here, an item is an “http://example.org/animals#cat”, and most of the properties have names that are words defined in the context of that type. There are also a few additional properties whose names come from other vocabularies.
<section itemscope itemtype="http://example.org/animals#cat">
 <h1 itemprop="name http://example.com/fn">Hedral</h1>
 <p itemprop="desc">Hedral is a male american domestic
 shorthair, with a fluffy <span
 itemprop="http://example.com/color">black</span> fur with <span
 itemprop="http://example.com/color">white</span> paws and belly.</p>
 <img itemprop="img" src="hedral.jpeg" alt="" title="Hedral, age 18 months">
</section>

The RDF equivalent to this is:

[
  a <http://example.org/animals#cat> ;
  <http://example.org/animals#name> "Hedral" ;
  <http://example.com/fn> "Hedral" ;
  <http://example.org/animals#desc> "Hedral is a male american domestic shorthair, with a fluffy black fur with white paws and belly." ;
  <http://example.com/color> "black" , "white" ;
  <http://example.org/animals#img> <hedral.jpeg>
]

To create this, we need the RDFa:

<section vocab="http://example.org/animals#" typeof="cat">
 <h1 property="name http://example.com/fn">Hedral</h1>
 <p property="desc">Hedral is a male american domestic
 shorthair, with a fluffy <span
 property="http://example.com/color">black</span> fur with <span
 property="http://example.com/color">white</span> paws and belly.</p>
 <span rel="img"><img src="hedral.jpeg" alt="" title="Hedral, age 18 months"></span>
</section>

Link Relations

Here is an example of a page that uses the vEvent vocabulary to mark up an event:

<body itemscope itemtype="http://microformats.org/profile/hcalendar#vevent">
 ...
 <h1 itemprop="summary">Bluesday Tuesday: Money Road</h1>
 ...
 <time itemprop="dtstart" datetime="2009-05-05T19:00:00Z">May 5th @ 7pm</time>
 (until <time itemprop="dtend" datetime="2009-05-05T21:00:00Z">9pm</time>)
 ...
 <a href="http://livebrum.co.uk/2009/05/05/bluesday-tuesday-money-road"
    rel="bookmark" itemprop="url">Link to this page</a>
 ...
 <p>Location: <span itemprop="location">The RoadHouse</span></p>
 ...
 <p><input type=button value="Add to Calendar"
           onclick="location = getCalendar(this)"></p>
 ...
 <meta itemprop="description" content="via livebrum.co.uk">
</body>

This example is interesting because it contains, in the natural markup of the page, a rel attribute with the value bookmark, which is used for links that go to the page or section of the page within which the link is found. In this case, it’s the page. The RDF that should be generated from the page is:

[
  a <http://microformats.org/profile/hcalendar#vevent> ;
  <http://microformats.org/profile/hcalendar#summary> "Bluesday Tuesday: Money Road" ;
  <http://microformats.org/profile/hcalendar#dtstart> "2009-05-05T19:00:00Z"^^xsd:dateTime ;
  <http://microformats.org/profile/hcalendar#dtend> "2009-05-05T21:00:00Z"^^xsd:dateTime ;
  <http://microformats.org/profile/hcalendar#url> <http://livebrum.co.uk/2009/05/05/bluesday-tuesday-money-road> ;
  <http://microformats.org/profile/hcalendar#location> "The RoadHouse" ;
  <http://microformats.org/profile/hcalendar#description> "via livebrum.co.uk"
] .

The following statement could legitimately be generated as well:

<> 
  <http://www.w3.org/1999/xhtml/vocab#bookmark> <http://livebrum.co.uk/2009/05/05/bluesday-tuesday-money-road> ;
  .

but the item representing the event should definitely not have the http://www.w3.org/1999/xhtml/vocab#bookmark property.

Achieving this without significantly changing the HTML markup is problematic in RDFa because RDFa uses the rel attribute to provide properties for the resources that it describes within the page, overloading its standard use in HTML which is to describe properties of the page or sections within the page. The following involves the least amount of repetition:

<body vocab="http://microformats.org/profile/hcalendar#">
 <div typeof="vevent">
  ...
  <h1 property="summary">Bluesday Tuesday: Money Road</h1>
  ...
  <time property="dtstart" content="2009-05-05T19:00:00Z" datatype="xsd:dateTime" 
        datetime="2009-05-05T19:00:00Z">May 5th @ 7pm</time>
  (until <time property="dtend" content="2009-05-05T21:00:00Z" datatype="xsd:dateTime" 
               datetime="2009-05-05T21:00:00Z">9pm</time>)
  ...
  <a rel="url" href="http://livebrum.co.uk/2009/05/05/bluesday-tuesday-money-road"></a>
  <a about href="http://livebrum.co.uk/2009/05/05/bluesday-tuesday-money-road"
     rel="bookmark">Link to this page</a>
  ...
  <p>Location: <span property="location">The RoadHouse</span></p>
  ...
  <p><input type=button value="Add to Calendar"
            onclick="location = getCalendar(this)"></p>
  ...
  <span property="description" content="via livebrum.co.uk"></span>
 </div>
</body>

Notes:

In the above, the typeof attribute has been moved onto a wrapper <div> that encompasses the entirety of the page because if it resides on the <body> element, it’s assumed to apply to the document itself rather than a blank node. An alternative mapping would use about="_:event" to create a blank node for the event.
There’s no way to avoid creating a statement for the rel="bookmark" link, so the best we can do is make sure that the statement is accurate, and relates the current document to the provided URI. Unfortunately, that means creating a separate element for the url property, repeating that URL within the page, and adding an empty about attribute; here I’ve used an empty <a> element to express the relationship; a <link> element would do the same job if it were allowed in flow content.
The <meta> element in the original has been mapped to an empty <span> element as it isn’t allowed in flow content without an itemprop attribute.

Microdata + RDF

Jeni — Sun, 31 Jul 2011 19:55:44 +0000

As part of the ongoing discussion about how to reconcile RDFa and microdata (if at all), Nathan Rixham has put together a suggested Microdata RDFa Merge which brings together parts of microdata and parts of RDFa, creating a completely new set of attributes, but a parsing model that more or less follows microdata’s.

I want here to put forward another possibility to the debate. I should say that this is just some noodling on my part as a way of exploring options, not any kind of official position on the behalf of the W3C or the TAG or any other body that you might associate me with, nor even a decided position on my part.

Simplifying RDFa

As I’ve said before, RDFa, in my experience, is complicated not primarily because of the whole namespaces/CURIEs issue but because its processing model tries to be too clever. RDFa was designed to largely fit in with existing markup and turn it into embedded data “just” by adding a few attributes here and there. Thus a simple image like:

<img src="photo1.jpg">

is first marked up to indicate that it’s an image:

<img src="photo1.jpg" typeof="foaf:Image">

then to provide its license:

<img src="photo1.jpg" typeof="foaf:Image"
  rel="license" resource="http://creativecommons.org/licenses/by/2.0/">

and finally to add a title:

<img src="photo1.jpg" typeof="foaf:Image"
  rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
  property="dc:title" content="A Pretty Picture">

all by adding attributes to the one <img> element. The trouble with this approach is that the rules about how statements are made become extremely complex, dependent on context (eg what other attributes are present, what the parent element has on it, what content it has) and default in ways that are hard to remember.

Even having written an RDFa parser, having written code to mark up documents with RDFa, having taught it, I still cannot write RDFa past a trivial example and be 100% sure that it will produce what I was aiming to produce.

If we were to look at really simplifying RDFa, rather than making cosmetic changes, we need to address this complexity. It would certainly mean backwards-incompatible changes, such as dropping the use of particular attributes and revising the way the processing model works, such that future RDFa processors couldn’t be used on RDFa 1.0. There are two possible ways of approaching this:

retaining some backwards compatibility, and aiming for a simplified subset of RDFa 1.0 such that RDFa 1.0 processor will still get the intended triples out of data marked up with RDFa 1.1
dropping backwards compatibility entirely and using completely different attributes, essentially creating a new language

I do not know which of these routes is the best one to take.

My instinct is that the first will be hard to do. For example, there are already certain simplifications in RDFa 1.1 — such as assuming an element with no datatype attribute is giving a string value rather than looking to see if there are any non-text-nodes in the content of the element — which lead to markup that will not be processed correctly by RDFa 1.0 processors. Perhaps that could be addressed by rewriting history: creating a RDFa 1.0 Second Edition that includes any changes that are needed to make a simple subset viable.

What I want to explore here is what the second route — using entirely different attributes from those currently used in RDFa 1.0 — might mean. I think that in this case the substantial difference between microdata and this new language would be support for that much-derided requirement: decentralised extensibility.

Adding Decentralised Extensibility to Microdata

As I discussed earlier in the week, microdata is simply not designed for use in a web where publishers might want to use multiple vocabularies to mark up the same thing for different consumers. This focus is very probably the right one for the majority of uses, where publishers address single consumers or everyone has standardised on a single vocabulary. It’s certainly an assumption that keeps the markup simple.

However, there is a larger data web out there. It’s not just browsers and search engines who might look for and process data embedded within a page. Unlike with HTML, those few, large consumers don’t have to understand a particular vocabulary for other consumers to get valuable information from it. If you operate in a world of multiple consumers with different requirements, you need decentralised extensibility. And support for decentralised extensibility is RDF’s niche as a data model, its unique selling point.

Given that a new language would have to use a different processing model from RDFa 1.0, I would suggest that it simply uses microdata’s as a starting point. Using attributes from RDFa 1.0 would only cause conflicts with RDFa 1.0 processors. Microdata processing is there, already defined, already implemented. It isn’t going to go away. And you know, it’s pretty good.

The ‘new language’ would then not so much a ‘new language’ as an enhancement on something that already exists. It would be a set of additions that augment the data that is generated from normal microdata processing with a few extra features that are useful in a world where there are multiple vocabularies for the same domain, where publishers have to provide data to multiple consumers, where an RDF view of data is useful. Call it microdata+RDF.

So what would we need to add? Well, there are three things, I think, that make microdata hard to use in a decentralised world, and make it hard to generate good RDF from microdata markup:

lack of support for multiple types
scoping of properties by type
lack of datatypes

We would need to find a way to add these for use within the RDF extracted from the microdata markup such that a basic microdata parser would still generate the same JSON, and such that microdata’s DOM API would work as specified in the microdata spec. So we can’t change the types of values that are possible in microdata’s attributes or how they’re interpreted in the DOM API.

Multiple Types

Because of the restrictions I just mentioned in not touching microdata itself, we can’t simply make itemtype take multiple URLs. We could rely on itemprop="http://www.w3.org/1999/02/22-rdf-syntax-ns#type" as a mechanism of providing types for use by RDF processors, but I think that the types of something is such a fundamental property that it makes sense to have a dedicated attribute.

I suggest itemclass. It would only be allowed on elements with an itemscope attribute and would take a space-separated set of values in exactly the same way as the itemprop attribute. The values would be turned into URIs in the same way as for the itemprop attribute, which I’ll describe below.

Microdata+RDF would add a method to the existing microdata DOM API to enable people to access items by class rather than their single type. So:

document . getItemsByClass( classes )
Returns a NodeList of the elements in the Document that create items, that are not 
part of other items, and that have one or more of the types or classes given in the 
argument.

The classes argument is interpreted as a space-separated list of classes.

Note that for simplicity, because they are interpreted in the same way within the RDF model, this returns items whose itemtype is listed in the argument list of classes as well as those whose itemclass is listed.

Within the DOM API, the itemClass IDL attribute on HTML elements would reflect the itemclass attribute.

The itemclass attribute would be ignored for the purpose of creating JSON from microdata, and only be used when creating RDF.

An example would be:

<li class="conference" itemscope itemid="/2011/oscon/"
    itemtype="http://schema.org/Event"
    itemclass="http://microformats.org/profile/hcalendar#vevent /vocab/Conference">
  ...
</li>

The JSON generated from this would look like:

{
  "type": "http://schema.org/Event" ,
  "id": "http://lanyrd.com/2011/oscon/",
  "properties": {}
}

The RDF would look like:

<http://lanyrd.com/2011/oscon>
  a <http://schema.org/Event> ,
    <http://microformats.org/profile/hcalendar#vevent> ,
    <http://lanyrd.com/vocab/Conference> ;
  .

Disambiguating Properties

To work with the RDF model, properties have to have URIs. We need to have a way of easily creating the URIs for the short-name properties without people changing their existing microdata markup.

Note: I’ve substantially revised this section following discussion with Philip Jägenstedt. Old text is struck through, new text underlined.

The way that this is done in RDFa 1.1 is through a vocab attribute, which provides a URI prefix that is concatenated to any short-name properties or types. ~~We could use the same approach here, but call the attribute itemvocab to fit in with the general method of naming attributes in microdata.~~ Using this with microdata would be tedious for users however, and it would be easy for the itemtype and itemvocab to get out of sync in weird ways.

itemvocab would only be allowed on elements with an itemscope. The scope of itemvocab would be limited to the item itself, so that it’s not forgotten when it’s needed, particularly in copy-and-paste scenarios. However, to make it easier to use I think it should probably be given a default value if it isn’t present, as follows:

Instead, the vocabulary for the properties could be identified as follows:

set vocab to the itemtype of the item if it is present, and the URL of the document if not
use a substring of vocab:
1. if vocab contains a #, the substring of vocab up to and including the #
2. otherwise, the substring of vocab up to and including its final /

For example, if you have:

<li class="conference" itemscope itemid="/2011/oscon/"
    itemtype="http://schema.org/Event"
    itemclass="http://microformats.org/profile/hcalendar#vevent /vocab/Conference">
  ...
</li>

then ~~itemvocab~~ the item vocabulary would default to http://schema.org/.

There could be an extra restriction that if itemtype is specified, itemvocab must be in the same domain as that type; that could help prevent the weird situation where in the generated RDF the properties would be interpreted as being in a completely different vocabulary from the itemtype.

~~Within the DOM API, the itemVocab IDL attribute on HTML elements would reflect the itemvocab attribute.~~

Note: the following example has been altered in place.

For example, take the following markup:

<li class="conference" itemscope itemid="/2011/oscon/"
    itemtype="http://schema.org/Event" 
    itemclass="SocialEvent BusinessEvent EducationEvent">
  <h3>
    <a itemprop="url" href="/2011/oscon/">
      <span itemprop="name">OSCON 2011</span>
    </a>
  </h3>
  <p itemprop="location" itemscope itemid="/places/portland/"
     itemtype="http://schema.org/Place">
    <span itemprop="name"><a href="/places/usa/">United States</a> / <a itemprop="url" href="/places/portland/">Portland</a></span>
  </p>
  <p class="date">
    <time itemprop="startDate" datetime="2011-07-25">25th</time>–
    <time itemprop="endDate" datetime="2011-07-29">29th July 2011</time>
  </p>
  ...
</li>

The vocabulary for the <li> element defaults to http://schema.org/ based on the value of the itemtype. The short-named properties and classes within that item are turned into URIs by pre-pending http://schema.org/ to their name. Similarly, the properties on the nested http://schema.org/Place are pre-pended with http://schema.org/Place/. The resulting RDF would be:

@prefix s: <http://schema.org/>

</2011/oscon/>
  a s:Event ,
    s:SocialEvent ,
    s:BusinessEvent ,
    s:EducationEvent ;
  s:url <http://lanyrd.com/2011/oscon/> ;
  s:name "OSCON 2011" ;
  s:location </places/portland/> ;
  s:startDate "2011-07-25"^^xsd:date ;
  s:endDate "2011-07-29"^^xsd:date ;
  .

</places/portland/>
  a s:Place ;
  s:url <http://lanyrd.com/places/portland/> ;
  s:name "United States / Portland" ;
  .

Note: see below for how the values are created in this example.

The JSON would be just the same as from a standard microdata processor; there’s no mapping to URIs for that output:

{
  "type": "http://schema.org/Event",
  "id": "http://lanyrd.com/2011/oscon/",
  "properties": {
    "url": [
      "http://lanyrd.com/2011/oscon/"
    ],
    "name": [
      "OSCON 2011"
    ],
    "location": [
      {
        "type": "http://schema.org/Place",
        "id": "http://lanyrd.com/places/portland/",
        "properties": {
          "name": [
            "United States / Portland"
          ],
          "url": [
            "http://lanyrd.com/places/portland/"
          ]
        }
      }
    ],
    "startDate": [
      "2011-07-25"
    ],
    "endDate": [
      "2011-07-29"
    ]
  }
}

Adding Datatypes

How to manage datatypes in RDF generated from microdata is something where the best approach is not at all clear to me. A couple of years ago I talked about some frustrations with RDF datatyping, and datatypes in RDF still frustrate me by being hard to use in sensible ways throughout the RDF toolchain. Nevertheless, it’s what we have.

The possibilities I can see for microdata+RDF are:

Use plain literals for everything, including URIs, equivalent to using strings as microdata does. This makes things simple for the publisher and keeps the markup in the page clean, but makes it difficult for consumers who are using RDF toolchains: they will usually have to do some kind of processing of the RDF generated from microdata+RDF to add appropriate datatypes to the values. There are two issues with this approach:
- I have a feeling that microdata+RDF processors will make up their own rules to add datatypes to the data extracted from a page (using rules like those described below and/or sniffing of values and/or using information from known built-in vocabularies), in an effort to add value for their users. But if different processors do that in different ways, we have an interoperability problem.
- In some vocabularies, the datatype of a value is not derivable from the property. The most important/common example of this is skos:notation, which uses values with different datatypes to supply different identifiers from different identification schemes for a given concept.
Assign datatypes based on the element type in the HTML. If the property value has come from a URL attribute, assume that it’s a resource rather than a literal; if the element is a <time> element, work out the datatype based on the syntax of the datetime attribute; otherwise assume it’s a string and give it a language in the case that one is specified. This gives some information but leads to a somewhat strange situation where you can mark up something as a date/time but not as a number.
Supplement the processing described in 2. with some basic datatype sniffing. Basically, if the value looks like a number or a boolean value then assign it a numeric or boolean datatype based on its syntax. This could reuse the rules for recognising different literals from Turtle. This wouldn’t be perfect; in particular, it would guess that strings that consist purely of numbers such as zip codes were numbers. I’m inclined not to go down this path.
Supplement the processing described in 2. with a itemvaltype attribute that takes a token from the list of built-in XML Schema Datatypes or the token ‘literal’. The ‘literal’ token would be used to override the normal processing of URL attributes in the case where those really should be literals rather than resources. In this design, it would be easy to create literals using one of the most usual datatypes, but not possible to use datatypes that are specific to a given vocabulary.
Supplement the processing described in 4. by allowing the itemvaltype to take either a token or a URL. The thing I don’t like about this design is that the token would be interpreted as being within the XML Schema Datatypes vocabulary rather than the vocabulary specified for itemvocab (used for tokens in itemprop and itemclass). This seems like it might turn into a source of confusion, but if we went the other way and had itemvaltype being interpreted based on itemvocab, it would be harder to give a value the more common datatypes such as numbers and boolean values.

My inclination, somewhat reluctantly as it’s the most complex, would be to use the last of these, because it provides for decentralised extensibility of datatypes, and support for decentralised extensibility is the core aim of these extensions. In other words, have a itemvaltype attribute that can hold either a token, which must be one of literal or the local name of an XML Schema datatype, or a URL. On a <time> element, this would default to the appropriate type based on the syntax of the value of the datetime attribute.

To be conformant, the itemvaltype would have to be an allowed value type for the properties given in itemprop and the value of the property must be a legal value for the datatype. (In keeping with the style of the microdata specification, the mechanisms for working out what value types are allowed and what the legal values are for non-XML Schema datatypes would be left undefined — a consuming application would look at the definition of the vocabulary.)

Within the DOM API, the itemValType IDL attribute on HTML elements would reflect the itemvaltype attribute. The value of itemvaltype wouldn’t change the types of the values returned by element.itemValue or in the JSON mapping from microdata; it would purely be used when generating RDF from that data.

For example, if someone started with some markup like:

<div itemscope itemtype="http://schema.org/AggregateOffer">
  Priced from: <span itemprop="lowPrice">$35</span>
  <span itemprop="offerCount">1938</span> tickets left
</div>

it might be supplemented with some type information like:

<div itemscope itemtype="http://schema.org/AggregateOffer">
  Priced from: <span itemprop="lowPrice" itemvaltype="http://schema.org/Price">$35</span>
  <span itemprop="offerCount" itemvaltype="integer">1938</span> tickets left
</div>

which would generate RDF like:

@prefix s: <http://schema.org/>

[] a s:AggregateOffer ;
  s:lowPrice "$35"^^s:Price ;
  s:offerCount 1938 ;
  .

(Note: Here I’m assuming that schema.org defines a http://schema.org/Price datatype which includes a currency and a number. They don’t currently.)

The JSON would still be:

{
  "type": "http://schema.org/AggregateOffer",
  "properties": {
    "lowPrice": [
      "$35"
    ],
    "offerCount": [
      "1938"
    ]
  }
}

Non-Additions

When I wrote a couple of years ago about what microdata can’t do, one of the things that I identified was not being able to express XML Literals. Having thought about this more, what’s actually missing isn’t to do with RDF, but is the ability to use the innerHTML of an element to provide a value for a property rather than its textContent.

For example, the description of an event might run over several paragraphs, or even in a single paragraph include other markup such as emphasised text, ruby markup, or links to additional information. People who are working from the DOM API can capture this information when they need it by getting the innerHTML of the element rather than its itemValue, but in the JSON mapping, the value is always the itemValue — the text content of the element.

So this is a general microdata simplifying limitation. I’d argue that we shouldn’t add any special handling to plug this hole at the microdata+RDF level. If it turns out that having values that contain markup is useful then it will be added to microdata, and the microdata+RDF mapping would then be extended to create rdf:XMLLiterals or HTML literals (for which there is no defined datatype in RDF at the moment) for such values.

Similarly, I haven’t said anything in this post about providing machine-readable values to override the text content of an element. There is an open bug about whether and how that capability might be added to HTML/microdata. I happen to think that it’s useful, but that utility isn’t limited to RDF processing. Whichever route is chosen there, I think it’s important to keep the property values used by basic microdata and microdata+RDF aligned.

Summary

To summarise, one direction that we could take in aligning microdata and RDFa would be to define an extension to microdata to add support for decentralised extensibility and the RDF data model. I think that would entail adding attributes such as:

itemclass to make it easy to define multiple types for an item
itemvocab and some default processing to provide nice mappings for short-name properties into URIs
itemvaltype and some default processing to assign datatypes to values

For publishers and consumers, a single language with optional extensions greatly simplifies the use of embedded data. Property names don’t have to be repeated or balancing acts made between different processing models.

RDFa proponents get a syntax that can be used to generate a natural RDF model against which they can build RDF-oriented APIs and map to other formats such as JSON-LD.

For microdata proponents, this approach doesn’t pollute microdata with requirements that they see as superfluous, and doesn’t change the behaviour of core microdata processors. Browsers, search engines and other consumers can continue to use the JSON output and only those who really want to support RDF need to do so.

I’m sure that there are things that I’ve missed in my outline above, issues that I haven’t thought of. But if there is to be any kind of convergence between microdata/RDFa, this layered approach seems to me to be the kind of convergence that is most likely to eventually result in one language for embedding data in HTML rather than two or three.

Note: if you prefer to comment on Google+, please add your comment to my announcement post there

My Experience of Web Standards

Jeni — Sun, 24 Jul 2011 16:24:00 +0000

One of the things that’s been niggling at the back of my mind since the schema.org announcement is how small a role search engine results plays in the wider data sharing efforts that I’m more familiar with in my work on legislation.gov.uk, and more generally how my day job experience differs from (what seem to be) more common experiences of development on the web. In this post, I’m going to talk about that experience, and about the particular problems that I see with the coexistence of microdata and RDFa as a result.

My day job (the one I actually get paid for) is web development. The site I spend most of my time and effort on is legislation.gov.uk. This deals with complex content (UK legislation) that has to be presented in multiple formats (users love PDFs of legislation). Our aim is to make the data as reusable as possible by third parties through good, RESTful, web architecture, and we want to use open standards and open source technologies as part of the UK government’s general strategy.

legislation.gov.uk is not a global website like Amazon or eBay, but it’s not small either: it covers 60,000 changing items of legislation, providing point-in-time views for many of them, and with more added every day. It’s one of the top ten most used UK Government websites, with 2 million visits (about 10-12 million page views) each month and typically about 120 requests/second during the active times of the day. Legislation might sound like a highly specialist interest, but if you search for legislation.gov.uk on Twitter you’ll see it being referenced over and over by people who want to share what the law says.

I do not by any means claim that my experience is representative of the wider web. I know that there are large numbers of sites that deal only in data, not documents, and certainly not documents with the kind of rich semantic structure that legislation has. I offer the following discussion as a data point, partly because I can’t quite believe that legislation.gov.uk is completely unique in its requirements and partly because obviously my perspective on a bunch of issues arises from this experience.

Technology Stacks

Legislation items are complex, semi-structured documents. Their natural fit is XML (well, that’s not quite true — their natural fit would be something that allowed overlapping markup — but XML is the closest that we have). So we store it in XML in a native XML database and we use an XML toolset to query it (XQuery) and transform it (XSLT) into various formats including rendering it as PDF (through XSL-FO).

Our next step for the development of the site involves looking at legislative effects. These form a graph: one item of legislation affects other items of legislation which may in turn affect other items and so on. There are all sorts of other links between items of legislation in terms of commencements, conferred powers and so on. Particularly because we already have well-thought-through URIs for legislation, the natural fit is to use RDF to represent this graph. We already offer a SPARQL endpoint for accessing some aspects of our data, but we expect to expand and develop this over the next few months and to use it as a layer under the website and exposed for reusers, in much the same way as we use the XML database.

As a government site, we have fairly strict limits on what we can do within our web pages: we have to make sure that they’re accessible by everyone who wants to view them. We aren’t able to use technologies that are only available in the latest browsers, but that’s OK because with the kind of content we deal with, we don’t have to do anything fancy anyway. So we use pretty basic HTML and CSS and Javascript, because that’s how you deliver content to end-users on the web (as well as exposing the underlying XML and RDF, to enable others to reuse the data).

In other words, we use three web stacks for delivering legislation.gov.uk:

the XML stack, which is great for single-source publishing of documents that have more semantic structures than those supported by HTML
the RDF stack, which is well-suited for metadata about things that are identified by URIs
the HTML stack, which is absolutely necessary for delivering human-accessible content on the web

What bemuses me, because of this experience, is that sometimes it appears that the narrative around these technologies is framed in terms of an exclusive choice between them. For example, @mattur asked:

It is as if, if you use XML you cannot appreciate the utility of error-handling in HTML; or if you use RDF you cannot understand the need to represent documents in XML; or if you want to utilise HTML fully, you cannot adopt RDF’s view of data on the web. That’s simply not my experience. They each have their role on the web; supporting the use of one does not necessitate rejecting the use of the others.

It’s interesting that some of the standards that are most reviled are those that arise at the intersections, where it appears that one technology is trying to encroach on the space of another:

XHTML at the border of XML and HTML
RDF/XML at the border of RDF and XML
RDFa at the border of all three

At the same time, within legislation.gov.uk, we publish XHTML (because it’s the natural output from an XML toolchain) and create and process RDF/XML (because it gives us access to that data from within the XML toolchain). We use a small bit of RDFa in the XHTML to indicate the rights under which our information is avaialble, and don’t yet, but are thinking about using RDFa to mark up non-document semantics within our XML (to enable the XML markup to focus on the document structures that it’s good at). For all their imperfections, these intersection technologies are useful for managing cross-overs; the problems arise when they overstep their remit and people start to think that all HTML must be XHTML or all XML must be RDF/XML or all RDF must be RDFa.

Sharing Scenarios

The second thing that I wanted to explore is the experience from legislation.gov.uk of what it’s like to be a publisher who actively wants to share their data. We need to operate simultaneously at three levels in our data sharing efforts.

Large-Scale Consumer-Driven Data Sharing

The first target for our data sharing efforts are the search engines. Obviously we’re not selling anything, but we want people to be able to locate legislation easily when they want it, and we want people who have done the search to be able to see some information about the legislation so that they know that they’ve located the right item.

This is large-scale consumer (search engine) driven data sharing, typified by schema.org and Facebook’s Open Graph Protocol (OGP). There are a few very big data consumers (Google, Microsoft, Yahoo!, Facebook etc) who need to consume data from large numbers of data providers. These consumers obviously can’t understand everything, so they determine and document what syntaxes and vocabularies they do understand and expect publishers to follow.

The benefits that publishers get from a particular consumer determines which syntax/vocabulary they use; publishers who are particularly keen to show up prettily within search results will target schema.org whereas those who want to be sharable within Facebook will target OGP. Many publishers will want to target both. There is probably a driver towards eventual convergence:

publishers might push back about inserting two lots of very similar data in their pages
consumers might want to include data from publishers who haven’t specifically targeted them

although there’s likely to be a period where they coexist, much as there was for VHS and Betamax (and V2000, I know, dad) during the early days of video players.

As I discussed previously, these large-scale consumers will be driven by the data that they find in the wild, in all its messy variety. They get relatively little benefit directly from using a generic syntax, as they are really interested in only a few, pretty generic, vocabularies for which they have hardwired processing. Indirectly, adopting a generic syntax has benefits in that publishers might find it easier to find tools that enable them to generate it, tutorials about how to use it, and feel that they aren’t being quite as locked in to something proprietary. However, rejecting data that isn’t marked up properly using that syntax has no benefit for consumers except in so far as it makes them feel that they are being good community members.

This is the pattern we see with schema.org (which accepts microdata but, based on its documentation, won’t reject data that isn’t fully compliant with it) and with OGP (which accepts a subset of RDFa but doesn’t reject data that hasn’t got prefixes properly bound, for example).

Another point to mention is that there is very little trust in this scenario. The communication between consumers and publishers is very limited, and the consumers will want to protect themselves against accidental or malicious errors that are evident in mismatches between explicit metadata and that which is parsed from the visible content of the page.

The parallels to HTML and browser vendors are very strong in this type of data sharing.

Small-Scale Consumer-Driven Data Sharing

A second type of data sharing is again driven by consumers, but this time at a lot smaller and more specialised scale. For legislation.gov.uk, these are services such as GLIN, which is a global legislation registry. Other examples are the recent work that we’ve done to publish UK Government organograms or Chris Taggart’s Open Election Data project. In these cases, there’s a single, relatively small and specialised consumer and a small number of publishers which are closely coordinated together.

As in the large-scale case, the consumer ultimately determines the syntax/vocabulary that it recognises, and communicates that to the publishers. However, small-scale consumers typically have close coordination with the publishers, which has a number of side-effects:

consumers may be more able to both apply pressure to and help publishers to do well in their markup
publishers have the opportunity to feed back directly to the consumer any suggestions that they have about changes to the syntax/vocabulary
publishers are likely to gain an immediate and tangible benefit from their cooperation, such as visualisations of their data that they otherwise wouldn’t have seen

Another noteworthy point about small-scale consumers is that they’re unlikely to have the engineering capability to create a custom parser for a particular syntax, but will instead want to use something off-the-shelf to extract data from pages and into their own backend systems. This, coupled with the closer coordination with publishers, means that they’re much more likely to stick to a specification, assuming that the off-the-shelf tools do.

Publisher-Driven Data Sharing

The final type of data sharing is driven by publishers. At legislation.gov.uk, we’re motivated to make our data available for reuse for transparency/accountability reasons (to help citizens understand the law), efficiency reasons (to help parliament and government departments to publish new legislation better) and economic reasons (to foster innovation in legal publishing). We don’t have any individual consumers in mind when we publish our data, but have found that simply by publishing it well, we foster reuse.

In this case, we as publishers are highly motivated to ensure that the data we publish is easily parsed with something off-the-shelf, since that lowers the barrier for potential consumers. Publishers like us are very likely to have unique, specialised, content and need to use a vocabulary that fits closely to our internal data structures as this lowers implementation cost. Consumers can also trust publishers like us: we simply have no motivation to lie in the data that we provide for reuse.

Mixed Markup

As I’ve outlined above, publishers like legislation.gov.uk need to target several potential consumers at the same time:

large-scale consumers such as search engines
small-scale consumers that provide us with a useful service
specialist consumers that are interested specifically in our data

We cannot use a single vocabulary for all these different purposes. (Well, we could write our own vocabulary and describe mappings to other vocabularies using RDFS, but search engines wouldn’t read it.)

We must therefore use a mix of vocabularies:

generic vocabularies about things that search engines care about
specialised vocabularies for particular small consumers
site-specific vocabularies for sharing our unique data

It’s repetitive, but it’s manageable so long as we have a syntax that enables us to say that an item of legislation is a http://scheme.org/CreativeWork and a http://purl.org/dc/dcmitype/Text and a http://www.legislation.gov.uk/def/legislation/Legislation and allows us to give multiple properties the same value.

The way things are going at the moment, we might well end up having to use multiple syntaxes on the same page, as some consumers understand microdata, others consume RDFa, and still others will parse microformats. This leads to more repetition: adding itemprop for microdata, property for RDFa and specialised class attributes for microformats. But worse (much worse), each of the syntaxes uses a different parsing model to create an entity-property-value structure, so not only do we have to learn substantially different markup patterns but our pages quickly become some kind of hideous polyglot mess trying to balance between them.

Looking Forward

As I said at the start, I’m fairly sure that my experience at legislation.gov.uk isn’t representative of the wider web, but I don’t have a clear idea about just how unrepresentative it is, in terms of technology use or motivations around data sharing. When I read my twitter stream or blogs, there’s a massive sampling bias, both in terms of who I follow and what I read, but also about who talks about what they’re doing. (I’m reminded of Jeff Atwood’s post on the Two Types of Programmers: the vast majority of web developers don’t make a noise about what they do.)

Taking part in web standardisation today often feels like being part on an ongoing cold war between distinct camps rather than a community working towards common aims. The underlying question seems to be “who’s side are you on?” Every decision and activity is cast as a victory or defeat. Time is wasted on attack and defence, or on raking over past slights and stupidities, rather than on progress. Valid criticism from outside a group cannot be listened to for fear of giving ground, cannot be made within a group where it seems like betrayal.

It is the Robbers Cave Experiment played out in web standards. As a psychologist, I find it fascinating. As a developer, and particularly one who doesn’t self-identify with any single group, it is frustrating. As a TAG member, trying to work for the longer-term good of the web, it is worrying, because situations of intergroup conflict lead to groupthink and non-optimal solutions.

As I described above, a non-optimal outcome seems to be the most likely result of the particular microdata vs RDFa conflict for us at legislation.gov.uk. While I know we are not generally representative, I believe that it will be similarly bad for other developers: publishers, consumers and tool implementers.

This is a problem for all who want to foster data sharing on the web using open standards; it is not one that any one group can fix on their own. It’s my hope that a balanced task force of individuals with a variety of experience and backgrounds can provide a focus for us all to work together to solve it. If we can’t, then we have let our prejudice and bias overcome our judgement.

TAG F2F, June 2011

Jeni — Fri, 17 Jun 2011 10:44:12 +0000

As you may know, I accepted an appointment to the W3C’s Technical Architecture Group earlier this year. Last week was the first face-to-face meeting that I attended, hosted in the Stata Center at MIT. As you can tell from the agenda (which was in fact revised as we went along) it was a packed three days.

What I intend to do here is to briefly report on the major areas that we discussed and give a tiny bit of my own personal take on them. In no way should any of what I write here be judged as revealing the official opinion of the TAG, it’s just me saying what I think, and I’m not going to go into anything in depth because they’re all incredibly gnarly and contentious topics and I’d not only be here all year but also end up in a tar pit.

Role of the TAG

Usefully for me as a newcomer, our first session was about the ongoing role of the TAG. The TAG occupies a unique position within the W3C. According to its charter it was set up

To improve the effectiveness of Working Groups, to reduce misunderstandings and overlapping work, and to improve the consistency of Web technologies developed inside and outside W3C

The TAG ultimately has three routes to do this:

by providing specific advice on issues that are brought to its attention
by writing documents on basic web architecture principles that go through community review, particularly through the general review of the W3C standards track and become Recommendations
by advising the W3C Director (Tim Berners-Lee) about what he should do on the extremely rare occasions when there are issues that he is supposed to adjudicate on

In none of these cases is there anything that binds the people receiving the advice of the TAG, or reading Findings or Recommendations made by the TAG, to accept them or do anything about them. The power and authority of the TAG depends solely on the quality and utility of its arguments, which is how it should be in my opinion.

Client-Side Application State

The first technical session was about client-side application state and was a review of the Identifying Application State draft that T.V. Raman began before he left the TAG and that Ashok Malhotra has been working on since. This should in the next few months or so be published as a TAG Finding (though it is currently on the Recommendation track).

This work is essentially about documenting the different ways in which you can identify application state within a URI, why that’s a useful thing to do, and some of the pitfalls of using hash URIs to do so. Most of the discussion was about details to do with wording within the document. One thing I thought particularly interesting was the point that URI-based application state is relevant in all ‘active content’, not just in HTML; for example, scripting in SVG or in PDFs bring the same considerations.

Buffer Bloat

Over lunch on Monday we listened to and discussed a presentation by Jim Gettys on buffer bloat. Basically (and all the errors here are introduced by me), TCP/IP is designed to route around network blockages, but it can only do so if it detects them quickly. When you have big buffers in place, as in the case of all modern operating systems and hardware, blockages aren’t detected quickly; they’re only detected when the buffers fill up. Then buffers empty and the data has to be sent again. The net result is that connections get really slow, not just for upload or download but for both, not just for you but for everyone using the network.

Jim talked about how this is exacerbated by the large amount of web traffic and the design of HTTP, particularly the lack of use of HTTP pipelining (whereby several HTTP requests and responses are sent over one long-term connection), because it leads to lots of small messages which can’t be handled effectively. There’s lots more about this on his blog.

Jim also talked about the failure of certificate authorities and how we should be looking at distributed protocols using digitally signed data, pointing us in particular to CCNx.

Fragment ID Semantics

First thing Tuesday was a session that I led on fragids, in particular the problems that are arising out of the mime type registration of +xml types (3023bis) clashing with those that are used for, say, images, and what happens when these come together in something like SVG.

The same issues arise whenever you have documents with types that ‘inherit’ fragid semantics from two directions. For example, XHTML documents are XML documents, so constraints on +xml mean you shouldn’t use interpreted fragids (eg hash-bangs) on them, but they are also ‘active content’ which makes interpreted fragids useful. Similarly, in linked data you shouldn’t really use a hash URI to mean a Person with a primary resource that provides as a response an XML document with embedded RDFa, because according to XML fragid semantics, such a URI should point to an XML element.

Basically the use of fragids has grown markedly outside their original scope and these situations aren’t really covered in the specs. I am now tasked to create a document that describes the issues and suggests ways forward. So that will be fun.

Telcon with IAB

The second session on Tuesday was a telcon with the IAB which has a similar role within the IETF as the TAG does within the W3C. This was a bit of a ‘getting to know you’ session, covering the work of the two groups on:

versioning and extensibility
security
privacy, including Do Not Track

and talking about opportunities to meet and work together on various topics like these.

URI Definition Discovery and Metadata Architecture

The afternoon session on Tuesday was spent on Jonathan Rees’s work on the Architecture of the World Wide Semantic Web, which covers, amongst other things, what people in semantic web circles call httpRange-14. At core, this is about the kinds of URIs we can use to refer to real-world things, what the response to HTTP requests on those URIs should be, and how we find out information about these resources.

Jonathan has put together a document called Providing and discovering definitions of URIs which covers the various ways that have been suggested over time, including the 303 method that was recommended by the TAG in 2005 and methods that have been suggested by various people since that time.

It’s clear that the 303 method has lots of practical shortcomings for people deploying linked data, and isn’t the way in which URIs are commonly used by Facebook and schema.org, who don’t currently care about using separate URIs for documents and the things those documents are about. We discussed these alongside concerns that we continue to support people who want to do things like describe the license or provenance of a document (as well as the facts that it contains) and don’t introduce anything that is incompatible with the ways in which people who have been following recommended practice are publishing their linked data. The general mood was that we need to support some kind of ‘punning’, whereby a single URI could be used to refer to both a document and a real-world thing, with different properties being assigned to different ‘views’ of that resource.

Jonathan is going to continue to work on the draft, incorporating some other possible approaches. It’s a very contentious topic within the linked data community. My opinion is while we need to provide some ‘good practice’ guides for linked data publishers, we can’t just stick to a theoretical ideal that experience has shown not to be practical. What I’d hope is that the TAG can help to pull together the various arguments for and against different options, and document whatever approach the wider community supports.

Can publication of hyperlinks cause copyright infringment?

The first session on Wednesday was another session that I led, discussing the Publishing and Linking on the Web draft that Dan Appelquist and I have been working on.

The aim of this document is to explain the tensions between terms that are commonly used in legal documents such as “possession”, “adaptation” and “distribution” and the way that publication works on the web, in which multiple servers may have copies of the same document (because they cache copies to make the ‘net go faster), automated agents may make changes to those documents (such as compressing or resizing documents, or merging Javascript) and people may refer others to those documents through linking.

We’re particularly keen to argue that linking to something is not the same thing as distributing it. The web’s power arises through its links, so it’s important that people are able to link to something without being worried about what happens when/if the domain they link to is taken over by something illegal.

Dan and I are going to continue to work on this document in response to various suggestions around organisation and terminology, with a view to getting some ‘friendly legal experts’ to look it over and then seeking wider review. The intention is for it to eventually become a Recommendation as this will give greater weight to it as a document for a legal audience.

API Minimisation and Client-Side Storage

There were then a couple of short sessions.

Dan talked about API Minimisation, which is the design principle that to increase privacy we should design APIs that enable people requesting information to say exactly what information they need, and return only that rather that everything known about a think. Dan’s put together an draft and should be calling for review for it soon.

Ashok then led discussion on client-side storage and we brainstormed around some of the architectural/design issues about which we might want to write if we were to put together a document. This work is at a very early stage.

TAG Priorities

After lunch, we had a session on TAG priorities where we discussed which of the various pieces of work that we’re doing should receive the most attention and had a quick review of who is doing what within the TAG.

Our basic problem is that a lot of this stuff feels quite urgent, and we want to be responsive, but with only 5-6 of us “actively involved” (which means 1 day/week) in drafting documents, and other TAG duties taking up our time, it feels like we have taken on too much work. Our focus for the next little while is going to be on responding to issues where our lack of response might either hold people up or cause longer term problems (for example the publication of contradictory mime type definitions), which means things like the document on publishing and linking on the web will need to bubble in the background rather than being the focus of activity.

HTML5 Last Call

Our final session, for which we were joined by Philippe Le Hégaret, was on the HTML5 Last Call documents. The TAG has raised various issues over the course of HTML5 development and want to follow up on how those issues have been addressed in the documents. Our role means that we’re responsible for making sure there’s consistency with other specifications, and that there isn’t anything that seems like it’s going to cause problems in the long term.

The part that we spent most discussion time on was the relationship between Microdata and RDFa. We talked about the precedents for having two specifications that do very similar things but with different approaches, such as CSS and XSL, and how this isn’t necessarily a bad thing so long as they don’t contradict each other and people can move between them easily (because they have the same conceptual foundations).

I’m going to save my opinion on this topic for another post. Suffice it to say that microdata and RDFa as currently specified don’t work well with each other but it’s not at all clear what the best path forward is. The TAG decided to recommend that the W3C set up a Task Force to look at what the best way forward might be.

Final Words

If you want links to the minutes of the TAG F2F, they’re available within the agenda or on separate pages for:

If you have anything to say on any of these topics, please send email to the TAG mailing list. Or you could comment here or email me directly if you like. Which leads me on to talking about what I’d like to do in the TAG.

One of the guidance notes for new members to the TAG says:

TAG members are elected or appointed not to represent their individual member organizations, but the Web community as a whole. We try to take that responsibility very seriously.

I do take that responsibility seriously. Web architecture has to be a combination of practice and theory, balancing approaches that work right now with a desire to not break anything long term. I do practical work developing web applications with HTML, CSS, Javascript, XML, RDF, XSLT, XQuery and so on and so on every day, but I know I don’t see all the difficult corners of the open web standard space: no one person can.

I can listen though, so that’s what I will try to do: listen, digest, reflect and act.

But I have limited resources. Unlike most of the members of the TAG, I am not employed by a large organisation that pays me for time I take on the work that I do for the TAG. The W3C kindly paid for my flights to and from F2Fs, but not hotels or expenses. I wouldn’t have taken this on if I wasn’t prepared to shoulder the financial burden, but if there is anyone out there who might sponsor my participation, I’d love to hear from you.

The HTML5 DOM and RDFa

Jeni — Thu, 24 Sep 2009 08:05:03 +0000

One of the fundamental disconnects between HTML5 and previous versions of HTML is the way in which you answer the question “what is the structure of this page?”. Things that make use of that structure, such as RDFa, need to take this into account.

An example is the document:

<html>
  <head><title>HTML example</title></head>
  <body>
    <table>
      <span>Example title</span>
      <tr><td>Example table</td></tr>
    </table>
  </body>
</html>

There are two different ways in which you might interpret the structure of this document. First, you might view the structure to be as it is written, with the <span> element as a child of the <table> element and therefore a tree that looks like:

+- html
   +- head
   | +- title
   +- body
      +- table
         +- span
         +- tr
            +- td

Second, you might view the structure of the page to be the DOM as it is constructed by an HTML5 processor, which will move the <span> out from the table due to foster parenting, giving the result:

+- html
   +- head
   | +- title
   +- body
      +- span
      +- table
         +- tr
            +- td

Which you view it as doesn’t really matter at this point, but it does when you start to introduce markup that infers information based on the structure of the page, such as RDFa. Let me introduce some RDFa markup to the document:

<html xmlns:dc="http://purl.org/dc/elements/1.1/">
  <head><title>HTML+RDFa example</title></head>
  <body>
    <table about="http://example.com">
      <span property="dc:title">Example title</span>
      <tr><td>Example table</td></tr>
    </table>
  </body>
</html>

Now, if you view the structure to be as written, the <span> element is within the <table> element, and is therefore viewed as talking about whatever it is that the <table> element is talking about, namely http://example.com. So the RDF that you will glean from this page is:

<http://example.com> dc:title "Example title"

On the other hand, if you view the structure to be that constructed by an HTML5 processor, the <span> element is not within the <table> element, and is therefore viewed as talking about whatever the document is talking about, namely the document itself. So the RDF that you will glean from the page is:

<> dc:title "Example title"

This isn’t exactly a new problem. There has always been the possibility of Javascript embedded within a page changing the page by moving or inserting elements, making the page that a non-browser sees fundamentally different from the one that a browser sees. This has been used by SEO people and spam merchants to get search engines to direct people to pages which mutated into something different when they were actually visited by a browser. And this eventually lead to those people who cared about interpreting meaning from the structure of pages (ie search engines) to at least go some way towards evaluating the Javascript within the page in order to “see” the page as a human would.

So it’s not a new problem, but it’s still a problem.

For those people trying to define how RDFa is interpreted in HTML5, there are several unpleasant alternatives:

Define RDFa as operating over an HTML5 DOM. This would make things easy for Javascript implementations in as much as they can rely on being used with HTML5 DOMs, ie in HTML5 browsers. But it raises the implementation burden for other implementations, such as those based on XSLT or a simple tidy-then-interpret-as-XML approach: essentially every implementation will need to include an HTML5 parsing library.
Define RDFa as operating over a DOM, but leave the creation of that DOM as implementation-defined. This effectively passes the buck (“it’s not our fault that HTML5 processors will construct a different DOM from XML processors”) but makes it hard to test implementation conformance and for authors to know exactly how their page will be interpreted. For example, an implementation that constructed a DOM with randomly rearranged elements would be entirely conformant despite producing completely different triples from one that took the elements in the original order.
Define RDFa as operating over a serialisation, with precise wording that describes how that serialisation is mapped into a tree structure that is walked to process the RDFa within the page. This approach will prevent implementations that use other methods of constructing trees from being conformant; depending on how it’s defined that might include XSLT implementations and/or Javascript implementations and/or implementations that use standard (XML-based) libraries for parsing the documents.

Personally I lean towards the second of these: defining RDFa as operating over a DOM but placing no constraints on how that DOM is created. It leaves open the possibility of Javascript implementations to work on the DOM they see, which may be radically different from the one seen by other processors due both to HTML5 reordering of elements and the dynamic modification of the page through Javascript. (Several people use rdfQuery to do before-and-after parsing of RDFa within a page, turning browsers into semantic editors, for example.) But it also lets conformant implementations be constructed in other ways for implementation ease or user needs, supporting the use of XSLT through GRDDL and the static crawling of content with minimal processing.

Perhaps the set of permissible methods of DOM creation could be listed to prevent completely random processing, but I expect that it will be effectively limited through social and technological pressures. Implementations that build DOMs in random ways aren’t going to be as useful (to their users) as those that build them in expected ways; it’s also going to be far easier to implement RDFa processors using standard parsing libraries.

The approach is not without its downsides, of course. XSLT is similarly defined as operating over a tree model, with the question of how that tree model is constructed left to the implementation. Most processors decided to construct the tree using standard XML parsing, but famously MSXML would strip certain whitespace-only text nodes from the tree (unless you specified a parsing flag telling it not to), leading to incompatibilities and user confusion.

My guess is that the same kind of thing will happen with RDFa processors. It could very well be the case that an author will:

check their RDFa in an RDFa validator that constructs a static HTML5 DOM, revealing one set of triples
be confused when they then use a Javascript RDFa library within their page and get a slightly different set of triples because of some Javascript embedded in the page that changes its structure
be further confused when a search engine that uses a tidy-and-interpret-as-XML approach gleans yet another slightly different set of triples and displays it in the search result

So if this approach were chosen, I would expect wording in the specification that required implementations to state the method they used to create the DOM (ie it should be implementation-defined rather than implementation-dependent) and that warned authors of the most likely causes of differences between implementations (such as tree modifications performed by HTML5 processors and Javascript within the page). I’d also like to see tools that take an HTML page and indicate the triples that it generates under different common DOM construction methods, so that authors can see the variation in how their documents might be interpreted.

HTML5/RDFa Arguments

Jeni — Fri, 21 Aug 2009 18:28:16 +0000

When I came back from holiday, I caught up with the recent discussions around RDFa and HTML5. It’s exhausting reading so many posts repetitively reiterating the positions of people who all have the best of intentions but fundamentally different priorities. And such a shame that so much energy is spent on fruitless discussion when it could be spent at the very least improving specifications, if not testing, implementing, experimenting or otherwise in some very minor way changing the world.

The particular thread’s subject was the use of prefixes, which are used to provide a shorthand for URIs, which are used to name properties such as http://xmlns.com/foaf/0.1/firstName. It’s unquestionable, really, that prefixes are a source of problems:

with copy-and-paste, because it’s easy to lose the bindings that are used to construct the full URI
in general understandability, because people just don’t get the idea that the prefix foaf could be used for something other than http://xmlns.com/foaf/0.1/ or that http://xmlns.com/foaf/0.1/ might be bound to a prefix other than foaf
when declared using namespace declarations, because of differences in interpretation between HTML and XHTML and for the general namespaces-in-content problems that we see in XML

I think it’s generally the case that the Semantic Web community, who are used to using syntaxes such as RDF/XML and Turtle which use prefixes all the time, judge these as being less disadvantageous than the members of the HTML5 community, who are much more in touch with and concerned about the “common user”.

But underlying the arguments about the costs of prefixes are arguments about whether these disadvantages are important enough to stop

giving people shorthands for URIs and/or
using URIs when naming properties and/or
using RDF as the data model for data on the web

Fundamentally, members of the Semantic Web (capital S, capital W) community take it as a matter of faith that the correct data model to use for expressing data about things both on and off the web is RDF. If there’s anything that defines the Semantic Web community, it’s that underlying assumption. (Well, probably: I’m sure there are still some who hanker after Topic Maps.)

Further, they think it is absolutely essential that if a property such as ‘first name’ is used within a page, you can GET a URI to find interesting information about the property. For example, with http://xmlns.com/foaf/0.1/firstName, you will get

  <rdf:Property rdf:about="http://xmlns.com/foaf/0.1/firstName" 
    vs:term_status="testing" 
    rdfs:label="firstName" 
    rdfs:comment="The first name of a person.">
    <rdf:type rdf:resource="http://www.w3.org/2002/07/owl#DatatypeProperty"/>
    <rdfs:domain rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
    <rdfs:range rdf:resource="http://www.w3.org/2000/01/rdf-schema#Literal"/>
    <rdfs:isDefinedBy rdf:resource="http://xmlns.com/foaf/0.1/"/>
  </rdf:Property>

which among other things includes a human-readable label that you might use in a display of information about things that have the property, and a statement about the domain of the property, from which you can tell that anything that has a http://xmlns.com/foaf/0.1/firstName property is a http://xmlns.com/foaf/0.1/Person.

Given that you need URIs for properties, members of the Semantic Web community generally think that you need a way to shorten those URIs to make them palatable to people who might be embedding metadata in their pages. The cost of that is that you have to use prefixes, with the disadvantages that I’ve outlined above, but it’s a cost that’s worth paying to gain resolvability and concision. And since it can be done in RDF/XML and Turtle and SPARQL and almost every other syntax in existence for expressing RDF, it seems unnatural not to be able to do it in the metadata embedded within webpages.

At an equally fundamental level, members of the HTML5 community are unconvinced that RDF is a necessary or useful model to use for data. They do not see how it offers significant advantages over Javascript object structures, for example.

Part of the reason for not being convinced of the utility of RDF is that members of the HTML5 community think it simply isn’t important for properties to be named with resolvable URIs. After all, microformats have illustrated that applications can derive meaning without having a machine-readable definition of the semantics of a property.

I haven’t heard them make these arguments, but they could also point out that there are vastly more mash-ups constructed with the developer having knowledge and understanding of the data that they are mashing up (and therefore requiring human-readable but not machine-readable definitions), than there are generic applications that could, or do, actually retrieve and reason with schemas or ontologies.

Some within the HTML5 community even think it is dangerous to use information from a property or class’s URI, because if the metadata within an HTML page can only be accurately understood when coupled with information from an external document, applications that are built on being able to locate that external information are in real trouble if (when) that document disappears, either temporarily or permanently.

For example, say that I have a page that contains the triple:

<http://www.jenitennison.com/#me> 
  <http://xmlns.com/foaf/0.1/firstName> "Jenifer"^^<http://www.w3.org/2001/XMLSchema#token> .

If it is the case that an application that resolves that triple also (a) retrieves the information available about the property and (b) reasons using it and any information derived from it, then having that triple in my page entails:

<http://www.jenitennison.com/#me> 
  a <http://xmlns.com/foaf/0.1/Person> ,
  a <http://xmlns.com/foaf/0.1/Agent> ,
  a <http://www.w3.org/2000/10/swap/pim/contact#Person> ,
  a <http://www.w3.org/2000/10/swap/pim/contact#SocialEntity> ,
  a <http://www.w3.org/2003/01/geo/wgs84_pos#SpatialThing> .

but this information can only be known by collecting the documents at:

http://xmlns.com/foaf/0.1/firstName
http://xmlns.com/foaf/0.1/Person (actually the same document as http://xmlns.com/foaf/0.1/firstName but I can’t know that)
http://xmlns.com/foaf/0.1/Agent (as with http://xmlns.com/foaf/0.1/Person, actually the same document as http://xmlns.com/foaf/0.1/firstName)
http://www.w3.org/2000/10/swap/pim/contact
http://www.w3.org/2003/01/geo/wgs84_pos (resolving this doesn’t actually tell me anything extra, but I have to do it all the same to check)

In these circumstances, if, say, http://xmlns.com/foaf/0.1/Person can’t be located (my connection dips out for a second or whatever) then suddenly the page’s metadata means something different, which is surely a problem in a Semantic Web.

Putting aside all that, even if you did need URIs for properties, the HTML5 community feels that the costs in usability of using prefixes to shorten URIs are simply too high to justify the benefit of concision. Why not simply use the full URI within an attribute?

In summary, you will not be able to persuade members of the HTML5 community that it’s worth paying the usability cost of prefixes until you have persuaded them:

that RDF has significant advantages as a way of modelling data over and above Javascript object structures
that the ability to resolve the URIs that are used to name properties is not dangerous, and can even be helpful
that it is ugly and tedious to have to use full URIs when naming properties

Really I’m just trying to draw attention to the fact that the HTML5 community has very reasonable concerns about things much more fundamental than using prefix bindings. After redrafting this concluding section many times, the things that I want to say are:

so wouldn’t things be better if we put as much effort into understanding each other as persuading each other (hah, what an idealist!)
so we will make more progress in discussions if we focus on the underlying arguments
so we need to talk in a balanced way about the advantages and disadvantages of RDF

or, in a more realistic frame of mind:

so it’s just not going to happen for HTML5
so why not just stop arguing and use the spare time and energy doing?
so why not demonstrate RDF’s power in real-world applications?

Google's RDFa Support

Jeni — Wed, 13 May 2009 21:30:31 +0000

I can’t reply to Henri Sivonen

@JeniT What’s wrong with http://rdf.data-vocabulary.org/rdf.xml ?

in 140 characters.

http://rdf.data-vocabulary.org/rdf.xml is the the RDF schema that describes the classes and properties recognised by Google’s rich snippets, which promises to provide richer information about search results than is available currently, in the manner of SearchMonkey.

So what’s so bad about this RDF schema?

Well, firstly, unlike what I just claimed, it’s not the RDF schema for Google’s rich snippets. If you look at an example like this one from their own help pages:

<div xmlns:v="http://rdf.data-vocabulary.org/" typeof="v:Person">
   <span property="v:name">John Smith</span>
   <span property="v:nickname">Smithy</span>
   <span property="v:url">http://www.example.com</span>
   <span property="v:affiliation">ACME</span>
   <span rel="v:address">
     <span property="v:locality">Albuquerque</span>
   </span>
   <span property="title">Engineer</span>
   <a href="http://darryl-blog.example.com/" rel="v:friend">
   <span property="v:name">Darryl</span>
</div>

you’ll see that the CURIE v:name resolves to the URI http://rdf.data-vocabulary.org/name. And that resolves to… well, take a look. The fact that you can’t resolve a property URI to a definition within a schema or ontology really undermines the idea of self-describing documents. To know anything about what http://rdf.data-vocabulary.org/name means you have to be human enough to go hunting for the documentation that describes it. Contrast with http://xmlns.com/foaf/0.1/name which, if you ask for RDF, takes you to ontology in which you can look for the definition of that resource, and find out its domain, range, a label for it and so on.

So the redirection’s screwed up. What about the schema itself? Well, it’s not that it’s bad in and of itself; it reflects a reasonable model of the world of products, reviews, people, organisations. The model’s obviously heavily based on those used within the microformats that Google is also supporting: hProduct, hReview, hCard and XFN, which in turn are based on existing standards such as vCard or on solid document analysis on existing behaviour. So it’s not surprising that it’s a reasonable model.

The problem is not so much what has been done as what hasn’t. I’m not completely surprised that there’s no support for FOAF — the model is sufficiently different from that of hCard to make it harder to use — but not reusing the existing RDF ontology for vCard smacks of either ignorance or arrogance. I’ve advocated before going ahead and building your own ontologies rather than waiting for community standardisation, but not doing a rudimentary search for previous work is going too far.

The schema itself is very sparse. It only uses RDF schema, rather than OWL, so it’s not surprising that it’s not particularly expressive, but even so a few labels would have been useful! The classes that have been introduced have no hierarchy (they are all subclasses of rdf:Resource), so properties that are shared between people and organisations, such as v:name, have a domain of all resources. And there’s no property subclass, so no way to automatically identify the appropriate property to use as a substitute for rdfs:label when displaying information about a person or organisation. From a technical standpoint, it seems like it was hastily thrown together by people who either don’t understand why self-description is important, or don’t care about other people’s use of the vocabulary.

But more than these, the reason that I am disappointed with what’s been shown so far is that Google is really missing the point of using RDF: its extensibility. It’s easy enough to parse a set of microformats; to build something useful around a small number of known vocabularies. It’s easy, but it’s limited. When I heard the buzz of “Google supporting RDFa” I expected it to support not just the syntax of RDFa, but its extensibility, because otherwise what’s the point?

The quote that scares me most comes from the interview with RV Guha and Othar Hansson:

RV Guha … It’s really important that everybody, as far as possible, use the same vocabulary. So Google is essentially going to be making an investment in sort of hosting a vocabulary that maybe is Google Services. …

because a single centralised vocabulary is just not going to be feasible. The web’s too large and diverse for that.

And I believe that Google, by working with the web and combining semantic markup with their formidable computing power and existing natural language understanding, could do so much more. Let me talk through a really simple scenario to illustrate.

Google publish their own RDF vocabulary for products (as they have done). Amazon look at the vocabulary and decide that it doesn’t include some information that’s useful and important specifically for books, such as the author of the book. So Amazon extend Google’s vocabulary by adding a new class that’s a subclass of google:Product, amazon:Book and introduce another property in their RDFa markup, amazon:author, giving it a range of amazon:Book.

When Google indexes Amazon’s pages, the RDFa parser sees that the resources that are described in the pages are described as amazon:Books. It doesn’t know what a amazon:Book is, so it resolves the URI, looks at Amazon’s ontology and finds the label and description of the class, and the fact that it’s a subclass of google:Product. The fact that an amazon:Book is a kind of google:Product means that it can be displayed just like other products. Perhaps Google even specify some other things that can be put in your ontology to supplement the rich snippet, like an icon, or a list of properties that are important. I don’t know. The point is that the ontology provides information to Google about what to do with this class, without Google having to invent and own the vocabulary.

Similarly, when the RDFa is parsed, Google generates triples for this new amazon:author property just as it does for the other Google ones. It recognises that it doesn’t know what it means, so it resolves the URI for the property and finds a schema. The schema includes a label for the property, “author” and a natural language description. Google uses the label in the display of the rich snippet; it processes the natural language description to disambiguate the label and translate it into other languages. Perhaps it can even assess the importance of the property, and whether it should actually be displayed at all, by understanding its description.

Anyway, back to earth. These are the kind of pipe dreams that I used to ridicule semantic web folk about back before they body-snatched me! So although I’m sure that Google could do so much more, there are some things to be thankful for:

They have made the first steps towards recognising semantic markup as a potentially useful source of information.
They haven’t gone and invent yet another syntax for encoding semantic information within HTML. (Well, at least this arm hasn’t.)
They have reused microformats.
They have developed a vocabulary for a few things that are really useful, even if we did have ontologies for some of it already.
They will now have a stake in answering the difficult questions around trust, confidence, accuracy and time-sensitivity of semantic information.

And, of course, this will encourage people who haven’t previously been interested to use semantic markup, which will make data easier to get at and more open to reuse and I believe will benefit the web as a whole.

What You Can't Do with HTML5 Microdata

Jeni — Wed, 13 May 2009 00:57:46 +0000

Update: Fixed a couple of errors in the microdata code.

The HTML5 microdata proposal has hit the web, just days before Google announced its support for RDFa (or at least one vocabulary encoded using RDFa attributes). These are, indeed, “interesting times” for the semantic web.

Now, if you’re one of those weirdos who want to embed RDF triples within your web pages, what you’re going to care about is whether you can use microdata to do it. Those of us who have been using RDFa in anger, rather than in toy examples, know that it can be hard to map a particular set of RDF statements onto HTML content. I thought I’d take a look to see just what it would be like to create particular RDF with the HTML5 microdata proposal.

Basics

On the face of it, you can express any triple in microdata because a triple like this (Turtle):

<http://www.example.com/subject> <http://www.example.com/property> <http://www.example.com/object> .

can always, and anywhere, be expressed with (HTML5):

<span item>
  <link itemprop="about" href="http://www.example.com/subject">
  <link itemprop="http://www.example.com/property" href="http://www.example.com/object">
</span>

while a triple like:

<http://www.example.com/subject> <http://www.example.com/otherProperty> "value" .

can be expressed with:

<span item>
  <link itemprop="about" href="http://www.example.com/subject">
  <meta itemprop="http://www.example.com/otherProperty" content="value">
</span>

Of course having to use all those long, repetitive URIs is a bit of a pain and bloats out the markup, but we’d never expect this to be hand-authored, right? Right? And what we really care about is that we can express the RDF.

It’s not just the URIs that are long-winded, by the way. RDFa manages to cram a lot into each element, whereas microdata usually requires separate elements. This is an example from the RDFa specification:

<img src="photo1.jpg"
  rel="license" resource="http://creativecommons.org/licenses/by/2.0/"
  property="dc:creator" content="Mark Birbeck" />

which produces the triples:

<photo1.jpg> xhv:license <http://creativecommons.org/licenses/by/2.0/> .
<photo1.jpg> dc:creator "Mark Birbeck" .

In HTML5, I think this has to be done with:

<span item>
  <img itemprop="about" src="photo1.jpg">
  <link itemprop="http://www.w3.org/1999/xhtml/vocab#license" 
        href="http://creativecommons.org/licenses/by/2.0/">
  <meta itemprop="http://purl.org/dc/elements/1.1/creator" 
        content="Mark Birbeck">
</span>

It’s a bit more tedious, but also more obvious what’s going on. Even after handling RDFa as much as I have, I still struggle to work out when, for example, an href attribute is providing the object for a statement, and when the subject. And if you look at the London Gazette RDFa, you’ll notice many occasions where empty <span> elements are used to provide the equivalent of the inline <link> and <meta> elements shown above. (In fact, as far as I recall earlier drafts of RDFa allowed <link> and <meta> elements to be used this too.)

From what I can see, though, there are two things that the microdata proposal in its current form can’t handle: datatyping and XML literals.

Datatypes

Datatypes are important in RDF. Values of properties are often not just strings, but dates, times, integers and so on. The microdata proposal mentions using the <time> element to create values, and has this example:

<div item>
 I was born on <time itemprop="birthday" datetime="2009-05-10">May 10th 2009</time>.
</div>

The triple that you’d want to create from this is:

<> <http://www.w3.org/1999/xhtml/custom#birthday> "2009-05-10"^^xsd:date .

which makes it plain that the value is a date. However, the definition of the mapping from microdata to RDF makes it clear that the triple that’s created is:

<> <http://www.w3.org/1999/xhtml/custom#birthday> "2009-05-10" .

In other words, the value is a plain literal, not a date.

In RDFa, the datatype attribute is used to indicate the datatype of the value, so you can do:

<div xmlns:custom="http://www.w3.org/1999/xhtml/custom#">
  I was born on <span property="custom:birthday" content="2009-05-10" datatype="xsd:date">May 10th 2009</span>
</div>

It would be easy enough to say that the value of a <time> element has the datatype xsd:date, xsd:time or xsd:dateTime dependent on the syntax of its datetime attribute, but there are other times that you want typed values. We’ve used strings (as opposed to plain literals), integers and years. I wouldn’t want to rule out the use of custom datatypes such as colours (and RDF permits these). The JSON mapping could, perhaps, use an appropriate object if there is one, and otherwise use just the string value without too much loss of power.

XML Literals

Arguably less important is the lack of support for XML literals, which are values that contain markup. The example in the RDFa spec is:

<h2 property="dc:title">
  E = mc<sup>2</sup>: The Most Urgent Problem of Our Time
</h2>

which generates the triple (Turtle):

<> <http://purl.org/dc/elements/1.1/title> "E = mc<sup>2</sup>: The Most Urgent Problem of Our Time"^^rdf:XMLLiteral .

RDFa allows you to force a value as an XML literal or a plain literal using the datatype attribute. Otherwise, if the element has any element children then it’s assumed to be an XML literal, and if not, a plain literal. I think the microdata proposal could adopt the same course of action. The JSON mapping could, perhaps, result in a value which is an array or some other container for a sequence of text and element nodes.

Final Thoughts

To my mind, the HTML5 microdata proposal is unacceptable in its current form because, unlike RDFa, it can’t be used to represent all the statements that you might want to represent. If those issues were fixed, there would be pros and cons between it and RDFa. Microdata is more long-winded, but more explicit. RDFa is more arcane but doesn’t swamp the content of the page to quite the same extent.

Like a lot of people, I would have far rather seen a proposal which didn’t reinvent the wheel, but how does the old saying go: “The great thing about standards is that there are so many to choose from.” If the microdata proposal stays the course, I only hope that we’ll see consumers supporting both it and RDFa so that producers can choose which to use rather than being forced to embed both within their pages.

rdfa

Microdata and RDFa Living Together in Harmony

Mapping Rules

Mismatched Features

Local Property Names

Interpretation of <time> Element’s datetime Attribute

Content Overrides

<link> and <meta> Elements in Flow Content

Identifiers without Types

Built-in Prefixes

Literals Including Markup

Itemref

Lists

Datatypes

Languages

Multiple Types

The src Attribute

Link relations

Possible Subset of RDFa

Possible Subset of Microdata

Guidelines for Vocabulary Authors

Choosing Between Microdata and RDFa

Final Words

Mapping RDFa to Microdata

Page Metadata

Base URI

Explicit Subjects / ItemIds

Items from the src Attribute

Additional Properties for Images

Nested Images

Types with Blank Nodes

Inherited Subject

Anonymous Nested Resources

Hanging Rels

Implicit Resources

Overriding Text Content

Language Support

Literals that Include Markup

The resource Attribute

Multiple Types

Mapping Microdata to RDFa

Unidentified Items / Blank Node Subjects

Values from the src Attribute

Values from the datetime Attribute

Nested Items / Object Properties

Itemref

Multiple Values

Creating Multiple Statements

Creating an rdf:Seq

Creating a rdf:List

Multiple Properties Sharing a Value

Types

Global Identifiers

Global Property Names

Link Relations

Microdata + RDF

Simplifying RDFa

Adding Decentralised Extensibility to Microdata

Multiple Types

Disambiguating Properties

Adding Datatypes

Non-Additions

Summary

My Experience of Web Standards

Technology Stacks

Sharing Scenarios

Large-Scale Consumer-Driven Data Sharing

Small-Scale Consumer-Driven Data Sharing

Publisher-Driven Data Sharing

Mixed Markup

Looking Forward

TAG F2F, June 2011

Role of the TAG

Client-Side Application State

Buffer Bloat

Fragment ID Semantics

Telcon with IAB

URI Definition Discovery and Metadata Architecture

Can publication of hyperlinks cause copyright infringment?

API Minimisation and Client-Side Storage

Interpretation of `<time>` Element’s `datetime` Attribute

`<link>` and `<meta>` Elements in Flow Content

The `src` Attribute

Items from the `src` Attribute

The `resource` Attribute

Values from the `src` Attribute

Values from the `datetime` Attribute

Creating an `rdf:Seq`

Creating a `rdf:List`