[OPEN-ILS-GENERAL] Importing holdings data

[Dan Scott]

On 13 January 2011 21:13, John Morris <jmorris at beau.org> wrote:
> On Thu, 2011-01-13 at 20:36 -0500, Jason Etheridge wrote:
>
>> > fingerprint: Eh?  So how would I generate this?
>>
>> There should be a trigger to handle this for you (maybe depending on
>> your version of EG).  It's used for grouping records with similar
>> titles into metarecords (think FRBR-ish, but for lumping different
>> formats and editions together).  See the metabib.metarecord table.
>
> Nope, don't see any trigger functions on the biblio table.  So for now
> assuming it either isn't important or whatever needs it will generate
> it.  Otherwise it can be fixed later.

John - this suggests that you're using 1.6, in which ingest (indexing
and all of that goodness) happens outside of the database.

This is why you run the MARC records through marc2bre.pl to generate
the BRE (biblio.record_entry) JSON objects; then you run those BRE
JSON objects through direct_ingest.pl to generate all of the JSON
objects representing indexes, fingerprints, etc. Then you run it
through (parallel_)pg_loader.pl to generate the SQL that you can load
into the database. If you don't run the direct_ingest.pl step, then
you're not going to be able to search for your records (well, you'll
be able to search for them, but you won't find any hits).

In 2.0, you can skip the direct_ingest.pl step because the database
automatically generates all of the indexes, fingerprints, etc as a
result of triggers.

> Thanks for filling in the other details.  Since in theory I now have all
> of the information needed all that remains is writing the code
> Hopefully that won't take very long.  Will be starting it tonight.

Good luck! Hopefully you have copy locations, circulation modifiers,
copy status, open circulation transactions, etc all covered too. If
you only have one copy location and very simple circulation rules then
those might not apply to you.

For your earlier statement (parsing holdings out of the MARC records),
this can work for simple cases; it was the basis of the import demo at
http://svn.open-ils.org/trac/ILS-Contrib/browser/import_demo/trunk .
I'd suggest that if you take this approach, that it might make sense
to build it into marc2bre.pl as another set of options. (You're
parsing the MARC records anyways, might as well generate your call
number (asset.call_number) and copy (asset.copy) entries at the same
time that you're generating the BRE objects, because you have the
all-important record ID in hand to correlate your call numbers to MARC
records).