Python extensions for XPath and XSLT

This document describes how to use Python extension functions in XPath and XSLT like this:

<xsl:value-of select="f:myPythonFunction(.//sometag)" />

and extension elements in XSLT as in the following example:

<xsl:template match="*">
    <my:python-extension>
        <some-content />
    </my:python-extension>
</xsl:template>

Contents

XPath Extension functions

Here is what an extension function looks like. As the first argument, it always receives a context object (see below). The other arguments are provided by the respective call in the XPath expression, one in the following examples. Any number of arguments is allowed:

>>> def hello(context, a):
...    return "Hello %s" % a
>>> def ola(context, a):
...    return "Ola %s" % a
>>> def loadsofargs(context, *args):
...    return "Got %d arguments." % len(args)

The FunctionNamespace

In order to use a function in XPath or XSLT, it needs to have a (namespaced) name by which it can be called during evaluation. This is done using the FunctionNamespace class. For simplicity, we choose the empty namespace (None):

>>> from lxml import etree
>>> ns = etree.FunctionNamespace(None)
>>> ns['hello'] = hello
>>> ns['countargs'] = loadsofargs

This registers the function hello with the name hello in the default namespace (None), and the function loadsofargs with the name countargs.

Since lxml 4.1, it is preferred to use the FunctionNamespace as a decorator. Either pass an explicit function name (@ns("countargs")), or just use the bare decorator to register the function under its own name:

>>> @ns
... def hello(context, a):
...    return "Hello %s" % a

Now we're going to create a document that we can run XPath expressions against:

>>> root = etree.XML('<a><b>Haegar</b></a>')
>>> doc = etree.ElementTree(root)

Done. Now we can have XPath expressions call our new function:

>>> print(root.xpath("hello('Dr. Falken')"))
Hello Dr. Falken
>>> print(root.xpath('hello(local-name(*))'))
Hello b
>>> print(root.xpath('hello(string(b))'))
Hello Haegar
>>> print(root.xpath('countargs(., b, ./*)'))
Got 3 arguments.

Note how we call both a Python function (hello()) and an XPath built-in function (string()) in exactly the same way. Normally, however, you would want to separate the two in different namespaces. The FunctionNamespace class allows you to do this:

>>> ns = etree.FunctionNamespace('http://mydomain.org/myfunctions')
>>> ns['hello'] = hello

>>> prefixmap = {'f' : 'http://mydomain.org/myfunctions'}
>>> print(root.xpath('f:hello(local-name(*))', namespaces=prefixmap))
Hello b

Global prefix assignment

In the last example, you had to specify a prefix for the function namespace. If you always use the same prefix for a function namespace, you can also register it with the namespace:

>>> ns = etree.FunctionNamespace('http://mydomain.org/myother/functions')
>>> ns.prefix = 'es'
>>> ns['hello'] = ola

>>> print(root.xpath('es:hello(local-name(*))'))
Ola b

This is a global assignment, so take care not to assign the same prefix to more than one namespace. The resulting behaviour in that case is completely undefined. It is always a good idea to consistently use the same meaningful prefix for each namespace throughout your application.

The prefix assignment only works with functions and FunctionNamespace objects, not with the general Namespace object that registers element classes. The reasoning is that elements in lxml do not care about prefixes anyway, so it would rather complicate things than be of any help.

The XPath context

Functions get a context object as first parameter. In lxml 1.x, this value was None, but since lxml 2.0 it provides two properties: eval_context and context_node. The context node is the Element where the current function is called:

>>> def print_tag(context, nodes):
...     print("%s: %s" % (context.context_node.tag, [ n.tag for n in nodes ]))

>>> ns = etree.FunctionNamespace('http://mydomain.org/printtag')
>>> ns.prefix = "pt"
>>> ns["print_tag"] = print_tag

>>> ignore = root.xpath("//*[pt:print_tag(.//*)]")
a: ['b']
b: []

The eval_context is a dictionary that is local to the evaluation. It allows functions to keep state:

>>> def print_context(context):
...     context.eval_context[context.context_node.tag] = "done"
...     print(sorted(context.eval_context.items()))
>>> ns["print_context"] = print_context

>>> ignore = root.xpath("//*[pt:print_context()]")
[('a', 'done')]
[('a', 'done'), ('b', 'done')]

Evaluators and XSLT

Extension functions work for all ways of evaluating XPath expressions and for XSL transformations:

>>> e = etree.XPathEvaluator(doc)
>>> print(e('es:hello(local-name(/a))'))
Ola a

>>> namespaces = {'f' : 'http://mydomain.org/myfunctions'}
>>> e = etree.XPathEvaluator(doc, namespaces=namespaces)
>>> print(e('f:hello(local-name(/a))'))
Hello a

>>> xslt = etree.XSLT(etree.XML('''
...   <stylesheet version="1.0"
...          xmlns="http://www.w3.org/1999/XSL/Transform"
...          xmlns:es="http://mydomain.org/myother/functions">
...     <output method="text" encoding="ASCII"/>
...     <template match="/">
...       <value-of select="es:hello(string(//b))"/>
...     </template>
...   </stylesheet>
... '''))
>>> print(xslt(doc))
Ola Haegar

It is also possible to register namespaces with a single evaluator after its creation. While the following example involves no functions, the idea should still be clear:

>>> f = StringIO('<a xmlns="http://mydomain.org/myfunctions" />')
>>> ns_doc = etree.parse(f)
>>> e = etree.XPathEvaluator(ns_doc)
>>> e('/a')
[]

This returns nothing, as we did not ask for the right namespace. When we register the namespace with the evaluator, however, we can access it via a prefix:

>>> e.register_namespace('foo', 'http://mydomain.org/myfunctions')
>>> e('/foo:a')[0].tag
'{http://mydomain.org/myfunctions}a'

Note that this prefix mapping is only known to this evaluator, as opposed to the global mapping of the FunctionNamespace objects:

>>> e2 = etree.XPathEvaluator(ns_doc)
>>> e2('/foo:a')
Traceback (most recent call last):
...
lxml.etree.XPathEvalError: Undefined namespace prefix...

Evaluator-local extensions

Apart from the global registration of extension functions, there is also a way of making extensions known to a single Evaluator or XSLT. All evaluators and the XSLT object accept a keyword argument extensions in their constructor. The value is a dictionary mapping (namespace, name) tuples to functions:

>>> extensions = {('local-ns', 'local-hello') : hello}
>>> namespaces = {'l' : 'local-ns'}

>>> e = etree.XPathEvaluator(doc, namespaces=namespaces, extensions=extensions)
>>> print(e('l:local-hello(string(b))'