Extension functions for XPath and XSLT
======================================

This document describes how to use Python extension functions in XPath and
XSLT.  They allow you to do things like this::

  <xsl:value-of select="f:myPythonFunction(.//sometag)" />

Here is how such a function looks like.  As the first argument, it always
receives a dummy object.  It is currently None, but do not rely on this as it
may become meaningful in later versions of lxml.  The other arguments are
provided by the respective call in the XPath expression, one in the following
examples.  Any number of arguments is allowed::

  >>> def hello(dummy, a):
  ...    return "Hello %s" % a
  >>> def ola(dummy, a):
  ...    return "Ola %s" % a
  >>> def loadsofargs(dummy, *args):
  ...    return "Got %d arguments." % len(args)


.. contents::
.. 
   1  The FunctionNamespace
   2  Global prefix assignment
   3  Evaluators and XSLT
   4  Evaluator-local extensions
   5  What to return from a function


The FunctionNamespace
---------------------

In order to use a function in XPath/XSLT, it needs to have a (namespaced) name
by which it can be called during evaluation.  This is done using the
FunctionNamespace class.  For simplicity, we choose the empty namespace
(None)::

  >>> from lxml import etree
  >>> ns = etree.FunctionNamespace(None)
  >>> ns['hello'] = hello
  >>> ns['countargs'] = loadsofargs

This registers the function `hello` with the name `hello` in the default
namespace (None), and the function `loadsofargs` with the name `countargs`.
Now we're going to create a document that we can run XPath expressions
against::

  >>> from lxml import etree
  >>> from StringIO import StringIO
  >>> f = StringIO('<a><b>Haegar</b></a>')
  >>> doc = etree.parse(f)
  >>> root = doc.getroot()

Done. Now we can have XPath expressions call our new function::

  >>> print root.xpath("hello('world')")
  Hello world
  >>> print root.xpath('hello(local-name(*))')
  Hello b
  >>> print root.xpath('hello(string(b))')
  Hello Haegar
  >>> print root.xpath('countargs(., b, ./*)')
  Got 3 arguments.

Note how we call both a Python function (`hello`) and an XPath built-in
function (`string`) in exactly the same way.  Normally, however, you would
want to separate the two in different namespaces.  The FunctionNamespace class
allows you to do this::

  >>> ns = etree.FunctionNamespace('http://mydomain.org/myfunctions')
  >>> ns['hello'] = hello
  >>> prefixmap = {'f' : 'http://mydomain.org/myfunctions'}
  >>> print root.xpath('f:hello(local-name(*))', prefixmap)
  Hello b


Global prefix assignment
------------------------

In the last example, you had to specify a prefix for the function namespace.
If you always use the same prefix for a function namespace, you can also
register it with the namespace::

  >>> ns = etree.FunctionNamespace('http://mydomain.org/myother/functions')
  >>> ns.prefix = 'es'
  >>> ns['hello'] = ola
  >>> print root.xpath('es:hello(local-name(*))')
  Ola b

This is a global assignment, so take care not to assign the same prefix to
more than one namespace.  The resulting behaviour in that case is completely
undefined.  It is always a good idea to consistently use the same meaningful
prefix for each namespace throughout your application.

The prefix assignment only works with functions and FunctionNamespace objects,
not with the general Namespace object that registers element classes.  The
reasoning is that elements in lxml do not care about prefixes anyway, so it
would rather complicate things than be of any help.


Evaluators and XSLT
-------------------

Extension functions work for all ways of evaluating XPath expressions and for
XSL transformations::

  >>> e = etree.XPathEvaluator(doc)
  >>> print e.evaluate('es:hello(local-name(/a))')
  Ola a

  >>> namespaces = {'f' : 'http://mydomain.org/myfunctions'}
  >>> e = etree.XPathEvaluator(doc, namespaces=namespaces)
  >>> print e.evaluate('f:hello(local-name(/a))')
  Hello a

  >>> xslt = etree.XSLT(etree.ElementTree(etree.XML('''
  ...   <stylesheet version="1.0"
  ...          xmlns="http://www.w3.org/1999/XSL/Transform"
  ...          xmlns:es="http://mydomain.org/myother/functions">
  ...     <output method="text" encoding="ASCII"/>
  ...     <template match="/">
  ...       <value-of select="es:hello(string(//b))"/>
  ...     </template>
  ...   </stylesheet>
  ... ''')))
  >>> print xslt(doc)
  Ola Haegar

It is also possible to register namespaces with a single evaluator.  While the
following example involves no functions, the idea should still be clear::
  
  >>> f = StringIO('<a xmlns="http://mydomain.org/myfunctions" />')
  >>> ns_doc = etree.parse(f)
  >>> e = etree.XPathEvaluator(ns_doc)
  >>> e.evaluate('/a')
  []

This returns nothing, as we did not ask for the right namespace.  When we
register the namespace with the evaluator, we can access it via a prefix::

  >>> e.registerNamespace('foo', 'http://mydomain.org/myfunctions')
  >>> e.evaluate('/foo:a')[0].tag
  '{http://mydomain.org/myfunctions}a'

Note that this prefix mapping is only known to this evaluator, as opposed to
the global mapping of the FunctionNamespace objects::

  >>> e2 = etree.XPathEvaluator(ns_doc)
  >>> e2.evaluate('/foo:a')
  Traceback (most recent call last):
  ...
  XPathSyntaxError: error in xpath expression


Evaluator-local extensions
--------------------------

Apart from the global registration of extension functions, there is also a way
of making extensions known to a single Evaluator or XSLT.  All evaluators and
the XSLT object accept a keyword argument ``extensions`` in their constructor.
The value is a dictionary mapping (namespace, name) tuples to functions::

  >>> extensions = {('local-ns', 'local-hello') : hello}
  >>> namespaces = {'l' : 'local-ns'}

  >>> e = etree.XPathEvaluator(doc, namespaces=namespaces, extensions=extensions)
  >>> print e.evaluate('l:local-hello(string(b))')
  Hello Haegar

For larger numbers of extension functions, you can define classes or modules
and use the ``Extension`` helper::

  >>> class MyExt:
  ...     def function1(self, _, arg):
  ...         return '1'+arg
  ...     def function2(self, _, arg):
  ...         return '2'+arg
  ...     def function3(self, _, arg):
  ...         return '3'+arg

  >>> ext_module = MyExt()
  >>> functions = ('function1', 'function2')
  >>> extensions = etree.Extension( ext_module, functions, 'local-ns' )

  >>> e = etree.XPathEvaluator(doc, namespaces=namespaces, extensions=extensions)
  >>> print e.evaluate('l:function1(string(b))')
  1Haegar

The second argument to ``Extension`` can either be be a sequence of names to
select from the module, a dictionary that explicitly maps function names to
their XPath alter-ego or ``None`` (explicitly passed) to take all available
functions under their original name (if their name does not start with '_').

The third argument takes a namespace URI or ``None`` (also if left out) for
the default namespace.  The following examples will therefore all do the same
thing::

  >>> functions = ('function1', 'function2', 'function3')
  >>> extensions = etree.Extension( ext_module, functions )
  >>> e = etree.XPathEvaluator(doc, extensions=extensions)
  >>> print e.evaluate('function1(function2(function3(string(b))))')
  123Haegar

  >>> extensions = etree.Extension( ext_module, functions, None )
  >>> e = etree.XPathEvaluator(doc, extensions=extensions)
  >>> print e.evaluate('function1(function2(function3(string(b))))')
  123Haegar

  >>> extensions = etree.Extension( ext_module, None )
  >>> e = etree.XPathEvaluator(doc, extensions=extensions)
  >>> print e.evaluate('function1(function2(function3(string(b))))')
  123Haegar

  >>> functions = {
  ...     'function1' : 'function1',
  ...     'function2' : 'function2',
  ...     'function3' : 'function3'
  ...     }
  >>> extensions = etree.Extension( ext_module, functions )
  >>> e = etree.XPathEvaluator(doc, extensions=extensions)
  >>> print e.evaluate('function1(function2(function3(string(b))))')
  123Haegar

For convenience, you can also pass a sequence of extensions::

  >>> extensions1 = etree.Extension( ext_module, None )
  >>> extensions2 = etree.Extension( ext_module, None, 'local-ns' )
  >>> e = etree.XPathEvaluator(doc, extensions=[extensions1, extensions2],
  ...                          namespaces=namespaces)
  >>> print e.evaluate('function1(l:function2(function3(string(b))))')
  123Haegar


What to return from a function
------------------------------

Extension functions can return any data type for which there is an XPath
equivalent.  This includes numbers, boolean values, elements and lists of
elements.  Note that integers will also be returned as floats::

  >>> def returnsFloat(_):
  ...    return 1.7
  >>> def returnsInteger(_):
  ...    return 1
  >>> def returnsBool(_):
  ...    return True
  >>> def returnFirstNode(_, nodes):
  ...    return nodes[0]

  >>> ns = etree.FunctionNamespace(None)
  >>> ns['float'] = returnsFloat
  >>> ns['int']   = returnsInteger
  >>> ns['bool']  = returnsBool
  >>> ns['first'] = returnFirstNode

  >>> e = etree.XPathEvaluator(doc)
  >>> e.evaluate("float()")
  1.7
  >>> e.evaluate("int()")
  1.0
  >>> int( e.evaluate("int()") )
  1
  >>> e.evaluate("bool()")
  True
  >>> e.evaluate("count(first(//b))")
  1.0

As the last example shows, you can pass the results of functions back into
the XPath expression.  Elements and sequences of elements are treated as
XPath node-sets::

  >>> def returnsNodeSet(_):
  ...     results1 = etree.Element('results1')
  ...     etree.SubElement(results1, 'result').text = "Alpha"
  ...     etree.SubElement(results1, 'result').text = "Beta"
  ...
  ...     results2 = etree.Element('results2')
  ...     etree.SubElement(results2, 'result').text = "Gamma"
  ...     etree.SubElement(results2, 'result').text = "Delta"
  ...
  ...     results3 = etree.SubElement(results2, 'subresult')
  ...     return [results1, results2, results3]

  >>> ns['new-node-set'] = returnsNodeSet

  >>> e = etree.XPathEvaluator(doc, None)

  >>> r = e.evaluate("new-node-set()/result")
  >>> print [ t.text for t in r ]
  ['Alpha', 'Beta', 'Gamma', 'Delta']

  >>> r = e.evaluate("new-node-set()")
  >>> print [ t.tag for t in r ]
  ['results1', 'results2', 'subresult']
  >>> print [ len(t) for t in r ]
  [2, 3, 0]
  >>> r[0][0].text
  'Alpha'

  >>> print etree.tostring(r[0])
  <results1><result>Alpha</result><result>Beta</result></results1>

  >>> print etree.tostring(r[1])
  <results2><result>Gamma</result><result>Delta</result><subresult/></results2>

  >>> print etree.tostring(r[2])
  <subresult/>

The current implementation deep-copies newly created elements in node-sets.
Only the elements and their children are passed on, no outlying parents or
tail texts will be available in the result.  This also means that in the above
example, the `subresult` elements in `results2` and `results3` are no longer
identical within the node-set, they belong to independent trees::

  >>> print r[1][-1].tag, r[2].tag
  subresult subresult
  >>> print r[1][-1] == r[2]
  False
  >>> print r[1][-1].getparent().tag
  results2
  >>> print r[2].getparent()
  None

This is an implementation detail that you should be aware of, but you should
avoid relying on it in your code.  Note that elements taken from the source
document (the most common case) do not suffer from this restriction.  They
will always be passed unchanged.