I just found an article titled Querying s-expressions in Common Lisp that Slava Akhmechet posted to defmacro.org last year. (Yes, I know--old news. I have a lot of catching up to do.)

The article got me excited because I am an XQuery programmer and I've always wanted to see something like XPath in Lisp and in Perl. (Thanks Slava!) I've even mentioned this to my peers. But I'm not talking about a library to manipulate, navigate, or query XML. Rather, like Slava, I'm thinking about a library to deal with native data structures (s-expressions in Lisp and nested hashes/arrays in Perl).

In my work, I do 2 kinds of things with XQuery:

1. Locate large records in a native XML database containing tens of millions of records. For this I need a highly optimized system that indexes everything and that uses XQuery, such as Mark Logic Server.
2. Take records (for example, the ones I located in Thing 1 above) and transform them to render the results to XHTML or to some other XML format (Web service output, for example).
Most of the code that I write is for Thing 2 above, which could be replaced easily with a Lisp library. A properly written one would probably work faster than the XQuery that I use. And, in Lisp, you'd have to implement only the XPath part, because all of the rest of the XQuery functionality is already implemented in the Lisp language itself, in a way that is far better than anything XQuery could ever achieve.

XPath supports some idioms that allow you to concisely identify a record or a set of records. Some of these can be easily emulated using Slava's code as a basis together with some additional code. I'm certain that Slava's code could be extended easily to make such idioms just as concise as XPath. For example, given the following s-expression, which I'll call 'node':

    (root
     (places
      (place
       (city "New Orleans")
       (state "Louisiana"))
      (place
       (city "Ashfield")
       state "Massachusetts"))
     (people
      (person
       (name "Donnie")
       (age 42))
      (person
       (name "Tracy")
       (age 47))
      (person
       (name "Nina")
       (age 6)))))

and the equivalent XML (let's call it $node), the XPath for retrieving the names of the adults that appear in the data structure is:

    $node/people/person[age ge 18]/name/text()

or

    $node//person[age ge 18]/name/text()

The way that I came up with for doing the same thing in Lisp (using Slava's s-query code) is:

    (mapcar (lambda (y) (cadar (s-query y '(name))))
            (remove-if-not (lambda (x) (> (cadar (s-query x '(age))) 18))
                           (s-query *xml* '(people *))))

I'd love to see a way to do this in Lisp using 1/3 of the code that I'm using above. Or rather, I want to be able to do in Lisp everything that XPath does, in just as concise a manner. It is the XPath that gives XQuery its strength in a native XML database environment. I'd love to be able to give Lisp the same capabilities.

If anyone knows of a library like the one I'm describing here, please let me know in the comments.