[OPEN-ILS-DEV] Metabib Keyword Indexing Issue

[Mike Rylander]

I believe I've fixed this in 1.6.0.0 and beyond.  The problem has to
do with space preservation when grabbing the text content of deeply
nested chunks of xml.  We're already normalizing date ranges by
removing the "-" between the years, so I've added a space before the
first year and after the second.  See:

http://svn.open-ils.org/trac/ILS/changeset/14821

which should apply cleanly to 1.4 as well.

To address existing keyword entries with this problem, the following
should do what you want:

UPDATE metabib.keyword_field_entry
  SET value = REGEXP_REPLACE(value, E'(\\w+)(\\d{4}) (\\d{4})', E'\\1
\\2 \\3','g')
  WHERE value ~ E'\\w+\\d{4} \\d{4}';

Please let us know if that does what you expect if you have a chance to test it!

--miker

On Thu, Nov 5, 2009 at 11:30 AM, Doug Kyle <dkyle at grpl.org> wrote:
> Dan Scott wrote:
>>
>> 2009/11/4 Doug Kyle <dkyle at grpl.org>:
>>
>>>
>>> We are seeing cases where a 600 tag with an author and dates such as
>>> this:
>>>
>>> 600 1 0. ‡aMarquette, Jacques, ‡d1637-1675.
>>>
>>> Has the first name and date run together in the
>>> metabib.keyword_field_entry
>>> table like so, jacques1637. Its only with that type of 600 tag in the
>>> keyword_field_entry table.
>>>
>>> Has anyone else seen this? and where would I look to fix it, could this
>>> be
>>> an xsl file problem?
>>>
>>>
>>
>> Hi Doug:
>>
>> Would you be willing to file a bug about this at
>> http://bugs.launchpad.net/evergreen - including the version of
>> Evergreen you're running and a sample bib record that produces the
>> problem?
>>
>> What I wrote in IRC yesterday was "mkfe comes from an xpath expression
>> against the MODS3.x (depending on your EG version) transform of the
>> bib record, as written up in the ModsParser.pm module and some entries
>> in the database", which should help us track down where the problem is
>> being introduced. It could be the MARC-to-MODS stylesheet that's the
>> issue, or it could be something else in the chain. Having a bib record
>> that would help us reproduce the problem would be a great help.
>>
>>
>
> Thanks Dan,
>
> Sure, I'll file a bug with bib.  It would be great if you or somebody else
> with the knowledge could provide an overview of the  ingest/metabib creation
> process and programs.  For example, I'm wondering if the metabib tables are
> all populated from the MODS transform?, is mfr done first and tables like
> mkfe populated from it?,  and where are the vector_index columns created?
> What code reading I've got done thus far leads me to think that if it was a
> MODS transform problem I would see the problem in various metabib tables for
> the record in question, but the same data is correct everywhere but mkfe.
>
> Doug Kyle
> Grand Rapids Public Library
>



-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com