[OPEN-ILS-DEV] Napkin drawings for removing OpenILS::Application::Storage::FTS

[Mike Rylander]

Search syntax thought experiment
--------------------------------------------------

Multi-stage search-to-query compilation

Given a default search class of "keyword" and a search string of:
  ( foo "bar" -baz || gar || ti|proper:qux) && au:junk


Stage 1: Boolean decomposition

{ bool : 'and',
  query : [
    { author : 'junk' },
    { bool : 'or',
      query : [
        { keyword : 'foo "bar" -baz || gar' },
        { 'title|proper' : 'qux' }
      ]
    }
  ]
}

Lacking grouping parens, we bind pairs of || or && separated atoms
(assuming &&), working from left to right.  If adjacent atoms belong
to the same class[|field] specifier, they are folded into a single
leaf.  Atoms are whitespace separated components not spelled '(', ')',
'||' or '&&'.



Stage 2: Search decomposition

{ bool : 'and',
  query : [
    { ftsquery : ['junk'],
      phrases : [],
      classname : 'author',
      fields : [7, 8, 9, 10]
    },
    { bool : 'or',
      query : [
        { ftsquery : [[['foo', '&', 'bar'], '&!', 'baz], '|', 'gar' ],
          phrases : ['bar'],
          classname : 'keyword',
          fields : [15]
        },
        { ftsquery : ['qux'],
          phrases : [],
          classname : 'title',
          fields : [6]
        }
      ]
    }
  ]
}

Then, we walk that tree (probably a plperlu stored proc) looking for
leaf hashes (those that don't contain a key of "query") and build up
SELECT (ranking), FROM (sourcing) and WHERE (tsquery and phrase
matching) clauses as we see them.  We apply the union of the index
normalizers defined for the referenced fields to the non-joiner atoms
in the "ftsquery" structure.

Eh?  I can see my way from here to there, but what I'm I totally overlooking?

-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com