[OPEN-ILS-DEV] ***SPAM*** Re: Apparent MARC import problem in 2.0.x

[Peters, Michael]

John,

You could always fall back to the "official" tools to load the bibs, since as best as I can tell from loading bibs in Evergreen Indiana, is working fine with 2.0.1.

-Mike

From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of John Craig
Sent: Wednesday, February 16, 2011 6:39 PM
To: Evergreen Development Discussion List
Subject: [OPEN-ILS-DEV] ***SPAM*** Re: Apparent MARC import problem in 2.0.x

OK. This problem is confirmed (as far as I can tell); and it's not a change between 2.0.0 and 2.0.1; I haven't traced back further than 2.0.0, but 2.0.0 has the same behavior (darn it).

In 2.0.x, somewhere in the Evergreen logic inside the open-ils.cat.biblio.record.xml.update method, my source records' 9xx tags are getting discarded--now, I expected this to happen with any 901 tags in the source data; but it's dropping all 9xx tags (907, 910, 945, 980, 998)

In response to Michael Peters' question from earlier today, yes, the tags are in the source records, but I'm not using marc2bre.pl. My program reads the records from a file and calls the open-ils.cat method via an OpenSRF session. (Just for the record, marc2bre.pl's output includes the 9xx tags.)

I've verified via the osrfsys.log file that the tags are in the XML parameter data passed to the open-ils.cat.biblio.record.xml.update method.

I don't know if it's even possible to configure PostgreSQL to capture the entire text of SQL commands--if it is, I do not have it set up properly. So, with Postgres logging data modification commands, although you can see the UPDATE biblio.record_entry statement, the text in the log file is too short to see if the data of interest is still there (the SQL is truncated; you can't see any of the bib record's tags at all--let alone the final ones). But, I've verified that the problem isn't arising in the in-the-DB-ingest logic by building an UPDATE statement and invoking it directly via  SQL. If the value passed to an UPDATE on biblio.record_entry.marc contains 9xx tags, they appear, as expected in the database's tables. (A SELECT on either the marc column itself or metabib.real_full_rec shows them.) But passing the same data to the open-ils.cat.biblio.record.xml.update method saves the bib data without the 9xx tags.

I've verified that the open-ils.cat.biblio.xml.update method is, in fact, updating the biblio.record_entry.marc value (by adjusting the content of the record that's updated via SQL vs what is passed to the open-ils.cat.biblio.record.xml_update method (so I can confirm that the method call is changing the data--and that the ingest trigger on biblio.record_entry is updating the associated metabib entries also). But somewhere between calling the method and the UPDATE on the database, my 9xx tags are being removed. (Or so I assume since an update on the DB with the 9xx tags works as expected--so this is technically an assumption.)

So, either there's a setting I'm not aware of (which would not surprise me), or there's a bug in open-ils.cat.biblio.record.xml.update or one of its subroutines. (Is there some kind of default associated with the logic that reads the trash_fields parameter of marc2bre.pl? So I'm losing these local tags because it's assumed I want to remove them?)

This would all be an interesting academic exercise, except we're supposed to migrate all this data this weekend....

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20110216/ceb3d5e9/attachment.htm