#-*-mode: org;-*-
* Purpose
"Oh, Ducks!" is an extension to cl-unification to make parsing
structured documents easy, using CSS selectors.
* Installation
** Prerequisites
 + cl-unification
 + cl-ppcre
 + split-sequence
 + alexandria
 + asdf-system-connections
 * closure-html
 * cxml
 * named-readtables
[+] Mandatory  [*] Optional
** Loading
Loading "Oh, Ducks!" is just like loading any other ASDF system.
However, because it does not mandate a particular HTML or XML parser,
it does not generally become useful until you have also loaded an
HTML/XML parsing library such as cxml or closure-html.

Start with:
: (asdf:oos 'asdf:load-op :oh-ducks)
If you would like to use the built-in support for parsing via
closure-html (which you almost certainly do), you'll also want to load
closure-html:
: (asdf:oos 'asdf:load-op :closure-html)
And, if you want to use DOM objects provided by cxml:
: (asdf:oos 'asdf:load-op :cxml)
** Load-order Caveats
closure-html and cl-unification each define competing readers on #t.
To avoid load-order issues resulting in an indeterminate reader on #t,
you'll probably want to add
: #.(set-dispatch-macro-character #\# #\T 'unify::|sharp-T-reader|)
or
: (unify:enable-template-reader)
or
: (named-readtables:in-readtable unify:template-readtable)
to the top of any file which uses cl-unification's reader templates.
(The latter two currently only work if you have cl-unification from my
darcs repo.)

Please feel free to submit patches to closure-html and cl-unification
to fix this problem.
** Depending Upon in ASDF Systems
It doesn't take long before managing your dependencies upon ASDF
systems becomes easiest by creating an ASDF system for whatever
project you're currently engaged in.  It's important to note that, in
addition to depending upon oh-ducks, you'll also want to depend upon
whichever library provides your desired object model and parser.

For example,
: :depends-on (:oh-ducks :closure-html :cxml)
** Differentiating between LHTML lists and XMLS lists
While it would, in theory, be possible to inspect lists and determine if they
are LHTML or XMLS lists, this is not currently done.  You can, however, choose
which type you'd like to work with by pushing =:lists-are-xmls= or
=:lists-are-lhtml= to =*features*= before loading "Oh, Ducks!".

Unfortunately, this means you can only expect to use one list type in a single
lisp image.  Patches to either automagically detect the list type, or to provide
layered functions are welcome.
* Usage
The combination of oh-ducks and closure-html provides an HTML template
for use with cl-unification, and has the following syntax:

  (match (#t(html [(:model <model>)]
                  <selectors>+)
          <document>)
    &body)
  selectors := (<selector> . <binding>) |
               (<selector> . <template>) |
               (<selector> <selectors>+)
  document := <parsed-document> | <document-to-be-parsed>

:model is only necessary for unparsed documents (e.g., a pathname or string).

** Examples

(match (#T(html (:model lhtml)
                ("#id" . ?div))
        "<div id=\"id\">I <i>like</i> cheese.</div>")
        (car div)) =>
  (:div ((:id "id")) "I " (:i () "like") " cheese.")

(match (#T(html (:model dom)
                ("i" . #t(list ?j ?i))
                ("span>i" . ?span))
        "<div>I do <i>not</i> like cheese.</div><div><span>I like <i>cheese</i>.</span></div>")
  (values i span)) =>
  #<ELEMENT i "not">,
  (#<ELEMENT i "cheese">)

** Selectors
The goal is to support all CSS-level-3 selectors.  See the section
[[*improve selector support][To Do > Improve Selector Support]] for a list of currently unsupported
simple selectors and combinators.

Each selector should result in the same elements which would be
affected by the same CSS selector.  That is,
  #id => elements with id of "id"
  .foo.bar => elements with both "foo" and "bar" classes
  div => all <div>s
and so forth.

NOTE: selectors are currently bound in parallel.  That is, given
  #t(html (<selector-1> ...)
          (<selector-2> ...))
selector-1 and selector-2 do not interact.  If they are both "foo", they'll
return identical results.  I often find myself wanting to also say something
like:
  #t(html (<selector-1> ...)
          (<element-after-selector-1> ...))
Ideas for a syntax to distinguish between the two cases are welcome (:mode
parallel) vs (:mode sequential), perhaps?  (Or even adjacent, sibling?)

*** Limitations

Currently, selector terms are limited to alphanumeric characters, and
do not support CSS-style character escapes.  Patches welcome!

** Included Object Models
*** LHTML (closure-html)
A list-based structure provided by closure-html.  Cannot be used by
selectors which require asking about parent or sibling objects.
*** PT (closure-html)
A structure-based structure provided by closure-html.
*** DOM (cxml)
DOM objects as provided by cxml and defined by the W3C.
* Extending
** Adding an object model
While the supported models should generally be sufficient, you can add
your own fairly easily.  All models are expected to implement the
generic functions in <traversal/interface.lisp>.  See the other files
under the traversal/ directory for examples.

You might also want to see chtml.lisp and cxml.lisp.
** Adding a selector or combinator
see <selectors.lisp>.  Generally, you should add a class which is a
subclass of combinator or simple-selector, augment parse-selector with
an appropriate regular expression, and define a method on
subject-p.

I also recommend submitting a patch.  Other people might want to use
that selector, too!
* To Do
** working lhtml/xmls support [2/2]
 * [X] non-descendant cases (class, id, etc.)
 * [X] selectors involving descendants
   CAUTION: Won't produce sane results if the document tree is
            modified or you use nested (match)es.
** write documentation
** improve selector support
*** positional selectors [11/11]
 * [X] :nth-child
 * [X] :nth-last-child
 * [X] :first-child
 * [X] :last-child
 * [X] :nth-of-type
 * [X] :nth-last-of-type
 * [X] :first-of-type
 * [X] :last-of-type
 * [X] :only-child
 * [X] :only-of-type
 * [X] :empty
*** attribute selectors [2/7]
 * [X] attribute-present  [att]
 * [X] attribute-equal    [att=val]
 * [ ] attribute-member   [att~=val]
 * [ ] attribute-lang     [att|=val]
 * [ ] attribute-begins   [att^=val]
 * [ ] attribute-ends     [att$=val]
 * [ ] attribute-contains [att*=val]
*** :not(...)
*** any others?
** namespace support(?)
** Submit patch to cl-unification to add (enable/disable-template-reader) functions
Submitted.  Was it ever accepted?  Man, I don't remember.
** Submit patch to closure-html to add (enable/disable-reader) functions
** non-css templates (e.g., for matching on text of element)?
Maybe special-case string/regexp-templates, so for example
: #t(html ("div" (#t(regexp "f(o+)bar") . ?div)))
would match [<div>foooobar</div>]?

: #t(html ("div" . #t(regexp "f(o+)bar" (?o))))
might cause some difficulty, however--we should get a list of matched elements
for the div selector, but the regexp variable (?o) can only match once (without
some wacky environment merging, anyway).
** Element structure templates
For instance, sometimes it'd be nice to stuff the value of an attribute into a
variable, like so:
:  (match #t(attr ("href" ?href) ("name" ?name)) "<a href='url' name='link'></a>"
:     (values href name)) =>
:    "url", "link"
While it's certainly easy enough to do that using, say, XMLS-style lists, a
general object-model-agnostic method would seem to be preferrable.
** Layered functions so LHTML vs. XMLS support can be switched at runtime