This is a package that makes it easy to write and run unit tests for
XSLT code. All you need is to unzip the .zip file unit-testing.zip somewhere and install Saxon 8.4B (or later), with
saxon8.jar in your CLASSPATH.
I've made various upgrades to the testing package since August. These changes are reflected in the text below.
My apologies to Linux and Mac users: I would supply shell files for
running the unit tests, but I don't know how to write them. Any kindly
soul who wishes to donate is more than welcome: it should all be
portable, since it's just Java and XSLT. Actually, I'm no expert at
writing Windows batch files either, so if anyone wants to smarten up
test.bat they're more than welcome.
I've tried to design this package to make writing tests easy. That means avoiding having to write anything that could be created automatically, such as test IDs (although you can supply them), calls to the tested functions or templates or comparison code. You write the tests inside your stylesheet, just before the code that you want to test.
This is a really simple example of a single test that checks the
result of calling the eg:square() function with the
number parameter set to 2. The
expected result is the number 4.
<test:tests>
<test:test>
<test:param name="number" select="2" />
<test:expect select="4" />
</test:test>
</test:tests>
<xsl:function name="eg:square" as="xs:double">
<xsl:param name="number" as="xs:double" />
<xsl:sequence select="$number * $number" />
</xsl:function>
All the testing elements are in the namespace
http://www.jenitennison.com/xslt/unit-test, so you need
to declare that in your stylesheet and you probably want to prevent
namespace declarations for that namespace littering your code, so you need
to use the exclude-result-prefixes attribute on
<xsl:stylesheet> as follows:
<xsl:stylesheet version="..."
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
...
xmlns:test="http://www.jenitennison.com/xslt/unit-test"
exclude-result-prefixes="... test">
...
</xsl:stylesheet>
The tests that are applicable to a particular template or function are
all wrapped in a <test:tests> element. This is
primarily to make it easy to collapse them all out of the way (in editors
that can do such a thing) so they don't become too distracting when
writing the code. Individual tests are given in
<test:test> elements.
Both <test:tests> and
<test:test> elements can have an
id attribute to specify a unique identifier and/or a
<test:title> child that gives a human-readable
title. Both are optional.
Within the <test:test> element, you need to
specify the input to the template or function and the expected output. The
input can consist of a context node (for templates only) specified with an
optional <test:context> element and any number of
parameters specified with <test:param> elements.
The expected output must be specified with a
<test:expect> element.
The values for the context, parameters and expected result are all
defined in the same ways. For atomic values (strings, numbers and the
like), use the select attribute, as in the example
previously. If you want to specify nodes, you need to supply a document
from which the nodes can be selected. You can do this either with the
href attribute, which gives a URI for an external
document, or by embedding the document (or document fragment) within the
relevant element. The select attribute is then used to
select nodes within the document; the path it holds is interpreted from
the root/document node of the document. The default value for the
select attribute is "/*", which
selects (all) the document element(s).
If you want to test the stylesheet as a whole, the easiest thing to do is to create external input/output files and reference them. Note that the result must be an XML document, so you can't use this to test HTML output.
<test:tests>
<test:test>
<test:context href="input.xml" />
<test:expect href="output.xml" />
</test:test>
</test:tests>
<xsl:template match="/">
...
</xsl:template>
You can test with simplified documents by simply embedding the
important part within the test itself. The embedded document should
include any important ancestors of the relevant element, but doesn't
need to include any of the irrelevant parts of the document. Use the
select attribute to pick the nodes you want from the
document. Remember that if your function or template returns something
other than elements, you will need a select
attribute on the <test:expect> element too.
<test:tests>
<test:test>
<test:title>Empty header cells</test:title>
<test:context select="/table/tgroup/thead/row/entry">
<table>
<tgroup>
<thead>
<row>
<entry />
</row>
</thead>
</tgroup>
</table>
</test:context>
<test:expect>
<th>Š</th>
</test:expect>
</test:test>
</test:tests>
<xsl:template match="thead/row/entry">
...
<xsl:when test="not(*) and not(normalize-space(.))">
<th>Š</th>
</xsl:when>
...
</xsl:template>
Quick version: change directory to the directory where you unzipped
anything and run test.bat with the stylesheet you want
to test as the only argument. The command line looks like:
test stylesheet.xsl
Long version: there are three steps, the last one optional, and of course you can reuse the results of any one of them.
Process your stylesheet with
generate-tests.xsl. The result is a stylesheet
that contains the code for the tests. You can put this wherever you
want; test.bat puts it in the same directory as the
original stylesheet, and calls it
test-stylesheet.xsl.
Generating the stylesheet shouldn't produce any errors, unless you've
written a test wrong or generate-tests.xsl still
contains bugs. I'm afraid that there's not much error-checking code in
generate-tests.xsl at the moment, so either could
result in errors.
Run the generated stylesheet using anything you like as the source
document, or by invoking the main template
directly. This runs the unit tests themselves. The result is an XML
document holding a report on what tests were run and their results;
test.bat puts it in the same directory again as the
original stylesheet, and calls it
test-stylesheet-result.xml.
You may get errors at this stage, associated with supplying invalid values for parameters for example. If the stylesheet fails any of the tests, you'll get a message telling you the ID and title of the test. The ID may be gibberish if you haven't supplied any IDs or titles, so this is a good reason to include them. If you don't get any errors or messages, then you know everything's gone smoothly.
Finally, particularly if there were failed tests that you can't
trace, you can format the XML report using
format-report.xsl. This produces a nice HTML page
(well, an HTML page) from the XML report; test.bat
puts it in the same directory as the original stylesheet, and calls it
test-stylesheet-result.html.
You shouldn't get any errors at this stage, but it's always possible.
Open up the HTML document and there you have it.
If you want to customize the HTML report that you get, you can edit
base.css or (of course),
format-report.xsl to your heart's content.
Wouldn't you know it, while I prefer to have tests embedded in my stylesheet, it turns out other people don't! Well, you can have the best of both worlds.
A standalone test suite has a <test:suite>
document element. The <test:suite> element has
two attributes: stylesheet, which is a URL (relative
to the test suite document) that points to the stylesheet tested by the
suite; and date, which is a xs:dateTime
that gives the date/time for the suite, useful for versioning.
The <test:suite> element contains one or
more <test:tests> elements, which are the same
as described above except that they also contain, immediately after the
<test:title> element if there is one, a
<test:xslt> element. The
<test:xslt> element contains either an
<xsl:template> or a
<xsl:function> element, with
name, match and/or
mode attributes to identify the template/function
being tested but no content.
Here's what a standalone test suite for a
utils.xsl stylesheet, last modified at 12:44 on
September 20th 2005, looks like. The only test shown is for the
eg:square function.
<test:suite stylesheet="utils.xsl"
date="2005-09-20T12:44:00">
<test:tests>
<test:xslt>
<xsl:function name="eg:square" />
</test:xslt>
<test:test>
<test:param name="number" select="2" />
<test:expect select="4" />
</test:test>
...
</test:tests>
...
</test:suite>
You can use a standalone test suite as the argument to
test.bat (or input to
generate-tests.xsl if you're doing things by hand),
and the tests will be run on the referenced stylesheet.
The extract-tests.xsl stylesheet transforms a
stylesheet that has tests embedded in it into a standalone test suite. I
haven't done the reverse of that as yet...
You can configure the details of how the testing is carried out, and
in particular how sequences/items/nodes are compared, by creating your
own implementations of the various functions in
generate-tests-utils.xsl. A
test:config attribute on the
<xsl:stylesheet> element, or a
config attribute on the
<test:suite> element in a standalone test suite,
can point to a stylesheet which contains these implementations in order
to override the default behaviour.
By default, this testing package compares the expected and actual results on an item-by-item basis. So if you expect
<xs:all> <xs:element ref="foo" /> <xs:element ref="bar" /> </xs:all>
but you get
<xs:all> <xs:element ref="bar" /> <xs:element ref="foo" /> </xs:all>
then the test fails.
In some cases, it might be that you really don't care what order
particular elements appear in, just as long as they're all generated.
In this example, the <xs:element> elements
appearing in the content of the <xs:all>
element can appear in any order with the same meaning.
To configure the testing package to ignore these ordering
differences, you can create xsd-config.xsl,
which overrides the test:sorted-children function
from generate-test-utils.xsl. This ensures that
the sequence of <xs:element> elements are
sorted by name before being compared, effectively ignoring the original
order in which they appeared.
To use xsd-config.xsl, include a reference to
it from the test:config attribute on the
<xsl:styelsheet> document element.
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:test="http://www.jenitennison.com/xslt/unit-test"
extension-element-prefixes="test"
test:config="xsd-config.xsl">
...
</xsl:stylesheet>
I've heard a lot about XP over the last few years, but never really got into the whole unit testing thing: I think that like a lot of programmers, I viewed writing tests as too much hard work. But during this summer (2005), I'm working on a project that's using an XP development methodology, plus Tim Bray talked glowingly about XP and unit testing during his talk about the XML Summer School. I decided that if I was going to do unit testing, I needed it to be really easy to write and run the tests.
So during the remaining couple of days at the XML Summer School, I hacked up the solution you see here. I tried to remain true to the rest of the XP philosophy: this does the minimum of what I need it to do at the moment, and if I find I need it to do more, then I'll add the code to do it.
The way I use it is that I've rigged up <oXygen/> with an
"External Tool" so that I can hit a button and have the tests run and the
results come up in the browser. If you use <oXygen/>, it's easy
enough to do: go to Tools > External Tools > Preferences, click New
and fill in the form as follows (I have the unit testing files in
D:\library\unit testing):
Test Stylesheet
Runs the unit tests embedded in the current stylesheet
D:\library\unit testing
"D:\library\unit testing\test.bat" ${cf}