creole

And she's back

Jeni — Sat, 11 Aug 2007 21:54:45 +0100

So first there was the XML Summer School. This year was my sixth, and it was really great to hang out with chums old and new. I love that

you get to meet people from all corners of the XML community, even ones you haven’t got the slightest interest in, and learn that they’re human too (even the web services guys)
there’s always something to learn; I’ve seen some talks for six years on the trot, others were completely new this year, but they’re all worth attending because the audience, war stories and discussion are always different. Also, because each talk is aimed at newcomers, you get a great overview of topics that you’re not so familiar with, and you can always chat to the speaker later to find out more
there are social events laid on every evening that you’re expected to attend, so you’re practically forced to socialise, which is useful for an insecure introvert like me who’d otherwise be sitting in her hotel room getting miserable imagining everyone else having a good time
there’s a creche, so despite being inseparable from two small children over the last four years, I’ve still been able to attend without dragging an entourage with me (not that I object to the entourage, just the expense and the dependency)

I left feeling not only invigorated and inspired, but also a part of a fun and friendly community.

The following week, we moved house. Twelve days later, we’re almost completely unpacked, and the important things are done. We have wireless, positioned nicely so that virtually the whole house gets “Excellent” coverage. Thanks to my father, we have a NSLU2 running Debian and acting as our low-power-consumption file and mail server. (For those that are interested, I’m getting access from my Windows machines using Xming to actually interact with the machine, and WebDrive to map a drive onto the file system.) And we have an area entirely dedicated to Lego. Yes, this might be my dream house.

(We’re also only a 40 minute drive from Heathrow, 30 minute train from Waterloo, so if anyone I know’s visiting the UK and wants to drop in, you’re more than welcome. There’s even a spare room.)

And amongst all this, I had to record a virtual presentation on Creole for Overlap day at Extreme. The sound quality’s not great, but it’s a reasonable 10-minute introduction, I think. It sounds like they got good attendance: will someone who was there please post about it?

Partitioning overlapping markup

Jeni — Mon, 11 Jun 2007 21:36:09 +0100

Wendell Piez forwarded me an interesting poster by Bert Van Elsacker on automatic fragmentation of overlapping structures. That’s taking something like:

<bold> this is bold <italic> and italic </bold> text </italic>

and turning it into something well-formed, like:

<bold> this is bold <italic> and italic </italic></bold><italic> text </italic>

When you do this, you have to decide which elements can be split and which can’t, and their relative priorities. Wendell suggested that perhaps Creole might help to do this. I have been thinking about is using Creole to add annotations to markup (something like, you add attributes to the Creole patterns and they get copied on to the matched ranges, or are used to create new ranges), but I haven’t done that yet, and actually I think you probably want a different kind of language to do it (a new kind of schema language like James Clark suggested), because the way in which you break up overlapping structures has a lot to do with how you’re going to process them.

I’m reminded of the paper

Sperberg-McQueen, C. M., David Dubin, Claus Huitfeldt and Allen Renear. “Drawing inferences on the basis of markup.” In Proceedings of Extreme Markup Languages 2002.

in which (based on my memory of the talk) they discuss how different elements allow you to make different assertions about the text they contain, and consequently can be split in different ways. For example, a <paragraph> element can’t be split into two <paragraph> elements without changing the meaning of the document, whereas a <bold> element can be split into two <bold> elements with no problems because it’s really indicating “these characters are bold” rather than “this is a bold phrase”.

You can take a purist view (which would usually entail splitting hardly any elements, since most elements do mark up a range of text rather than the individual characters they contain), but I think the main reason you want to do this fragmentation is for presentation. And in that context, the notional semantics of the element don’t really matter: what matters is how they’re styled. For example, a <comment> element, marking up a range of text that has been commented on, might not be splittable at a theoretical level, but if you’re going to render it simply by turning the background yellow, then in fact you can split it for that purpose.

Since it’s related to presentation, I wonder whether you could use a (simplified) CSS stylesheet to provide both the fragmentation and the style. Block-level elements (display: block;) couldn’t be split whereas inline elements could. Elements that have the box model properties (margin, padding & borders) can’t be split, or, if they are, you need to mark the fragments as “left”, “middle” and “right”, and only apply the left margin/padding/border to the “left” fragment, and similarly with the right.

It wouldn’t be a general purpose transformation mechanism, but it would be darned useful!

XTech Creole presentation fallout

Jeni — Wed, 16 May 2007 21:48:47 +0100

Henry Thompson had a lot to say after my Creole presentation (open takahashi.xul?data=creole.data; requires Firefox) about the benefits of stand-off markup for linguistic information. From his overview, it seems that the NITE XML Toolkit that he’s been involved with represents overlapping linguistic data by holding atoms (here meaning the “lowest common denominator” shared pieces of data) and having multiple trees marking up these atoms. The trees are independently validated (since they are pure XML), with cross-hierarchy validation done through the query language. This is pretty similar to the XCONCUR approach, which augments a CONCUR-like multi-grammar validation with a Schematron-like constraint language.

Now, I have nothing against using constraint languages (like Schematron) to validate documents, but grammars (like RELAX NG) have big advantages. Most importantly, they are easier to write (if they’re designed properly), and tools can analyse them to do useful things, such as tell you what element or attribute is expected next. If it’s possible to write cross-grammar constraints in a grammar (like Creole) then why would you use a constraint language to do it?

I think the big difference between Henry’s domain and the one that I think will move overlap into the mainstream is between global and local concurrence. With global concurrence, entirely separate hierarchies are applied to the same data, so the natural validation mechanism is to use entirely separate grammars (with perhaps a few small rules to do cross-grammar validation where that proves necessary). With local concurrence, the vast majority of the document follows a single hierarchy with concurrence happening at a low level.

Actually, the best example for this doesn’t even involve overlap. Consider HTML paragraphs, which contain various inline elements such as ,  and <a>. It doesn’t make sense for these elements to contain themselves (strong text is neither made stronger nor negated by appearing in two  elements, and it’s not allowed for links to contain other links). So the natural model in Creole is

p      = element p { mixed { strong* & em* & a* } }
strong = range strong { text }
em     = range em { text }
a      = range a { attribute href { text }, ..., text }

This model allows <a> elements to appear within  elements, or vice versa, not because of the content model of  but because the two ranges are interleaved (and one arrangement of interleaved ranges is containment). It doesn’t allow any of these elements to appear inside themselves. It would be a real maintenance headache to have separate grammars for each of these inline elements, when most of each of the grammars (all the hierarchy down to the paragraph level) would be the same.

Actually, looking at NITE, it seems like it employs a data model that’s quite like LMNL’s, in that it has the concept of layers over atoms or ranges/elements. (Interestingly it looks like they get around the problem of identifying which ranges belong to which layers purely by using their name.) Another difference here might be that while I’m talking about supporting overlap in fairly heavily structured documents (like office documents), they’re really using fairly flat annotations, where there isn’t much of a grammar anyway. But I might have that wrong: need to do more reading. The other thing to investigate is whether they have any support for self-overlap (<phrase> elements overlapping other <phrase> elements): I kinda gather that they don’t.

Anyway, Henry also made the points that (a) that he doesn’t want a new syntax for overlap and (b) stand-off markup works very well thank you. To address the latter point first, I think stand-off markup works very well if you have the tools to support it. It’s fine if you have an integrated toolkit which can pull together and display the stand-off markup as embedded markup, and let you create ranges by highlighting text with a mouse. But the great power of HTML and other web technologies is that you don’t need to use a specialised toolkit to write it: you can just use a text editor and it’s all right there in front of you with no (or minimal) cross-referencing required. Frankly, I’m not interested in “core” technologies that require me to install a particular piece of software in order to make use of them (cf Erik Meijer’s talk on LINQ, which I’ll have to discuss another time). I expect to be able to write a document containing overlap as easily as I can write a normal XML document.

On Henry’s point about yet another syntax for overlap, I am more and more coming to the conclusion that overlap will hit the mainstream if we have a simple way of encoding overlap in normal XML documents, namely something along the lines of LIX. Interestingly, Yves Savourel’s talk on Applying the Internationalization Tag Set was quite inspirational in this regard, since the working group seem to have put together a standard that both provides a set of standard elements and attributes to guide localisation, along with a method of mapping elements and attributes in existing markup languages onto those ITS elements and attributes. I wonder whether a similar approach could be used with LIX… but I’ll have to leave those thoughts for another time.

A Creole by any other name...

Jeni — Wed, 25 Apr 2007 21:09:28 +0100

Argh. I’ve been contacted by the guys at WikiCreole who want me to change the name of Creole. What should I do? Not only is “Creole” a great name for a schema language that deals with concurrent markup, but it’s a great acronym too (Composable regular expressions for overlapping languages etc.)

I did Google when I first came up with the name in August 2006, but didn’t discover WikiCreole (unsurprisingly, since it was only coined in July 2006 itself). But now far more many people know, care about and use WikiCreole than Creole grammars. So any suggestions for alternative names?