[OPEN-ILS-DEV] Add "sfr" class to fm_IDL.xml ?

[Dan Scott]

On Mon, 2009-12-21 at 12:01 -0500, Dan Scott wrote:
> On Mon, 2009-12-21 at 10:44 -0500, Warren Layton wrote:
> > Hello again,
> > 
> > I'm trying to run through the MFHD test dataset located in trunk:
> >   http://svn.open-ils.org/trac/ILS/browser/trunk/Open-ILS/tests/datasets/README
> > 
> > The first step, importing serials_marc21.xml, works fine. However,
> > when I try to import serials_mfhd.xml, I get the following error:
> > 
> >   Couldn't process record: Can't locate object method "new" via
> > package "Fieldmapper::serial::full_rec" (perhaps you forgot to load
> > "Fieldmapper::serial::full_rec"?) at
> > /openils/lib/perl5//OpenILS/Application/Ingest.pm line 1222.
> > 
> > Looking through fm_IDL.xml, I notice that there are classes "full_rec"
> > classes for metabib and authority records ("mfr" and "afr",
> > respectively), but not serials:
> > 
> >   <class id="mfr" controller="open-ils.cstore"
> > oils_obj:fieldmapper="metabib::full_rec"
> > oils_persist:tablename="metabib.full_rec" reporter:label="Flattened
> > MARC Fields">
> >   <class id="afr" controller="open-ils.cstore"
> > oils_obj:fieldmapper="authority::full_rec"
> > oils_persist:tablename="authority.full_rec" reporter:label="Full
> > Authority Record">
> > 
> > Should there be an "sfr" class, too? Or does something else (in
> > Ingest.pm?) need to be changed?
> 
> Hmm. This is probably a victim of my last-minute decision to pull the
> overhead of serial.full_rec classes out of the schema
> (http://svn.open-ils.org/trac/ILS/changeset/13680), because we weren't
> actually doing anything with them for serials.
> 
> It sounds like I failed to make the required corresponding changes to
> Ingest.pm. I'll poke at it this afternoon.
> 
> Thanks for bringing this up, Warren.
> 

So, a gross hack (that makes things work as well as they did before I
pulled the full_rec chunk of schema) is to patch Ingest.pm per the
attached patch; rather than adding sfr back, which would be a schema +
IDL change (not good in a stable release series), it causes the ingest
process to simply skip the full_rec processing step when bulk-loading
with marc2sre.pl. The other serial ingest methods would still break,
mind you, but we don't hit those in the current code (and it seems that
you're only the second site to even try to use this code).

Note that when you try to load the records into the system, you'll find
that you're missing the sre.owning_lib parameter. This has nothing to do
with full_rec; when I loaded our serial records, I munged the SQL output
manually to add that metadata.

The pg_loader output will look like:

COPY serial.record_entry
(active,record,create_date,creator,deleted,edit_date,editor,id,last_xact_id,marc,source,owning_lib) FROM STDIN;
t	814607	now	1	f	now	1	1	IMPORT-1261419747.17566	<record>...</record>	\N
\N

I munged that output by changing the final "\N" to the desired org_unit
ID to satisfy the NOT NULL constraint on the owning_lib column. You
could do that, or set a default value on the column, or rework the
ingest process to generate that extra metadata.

The challenge is in providing a standard mapping of serial record to
library, given that the MFHD specification and implementations in the
wild vary. And it would arguably be better to go via serial.record_entry
-> asset.call_number -> actor.org_unit rather than directly from
serial.record_entry -> actor.org_unit if we're going to rework the
ingest process.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: sfr_gross_hack.diff
Type: text/x-patch
Size: 802 bytes
Desc: not available
Url : http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20091221/6959970f/attachment.bin