[OPEN-ILS-DEV] MARC Record Import from txt

[Elliot Voris]

This has been beyond troubling!

Ideally, I would like to insert everything using the sql file and the cli. However, that's when I get the error message saying "non-standard use of escape in a string literal." No matter what I tried, I couldn't get past that.

Either way, I've got my marc records in my staging table now. I figured I could insert into the record_entry table, using standard SQL commands. In my mind, it should just insert into the table like it's supposed to. However, I'm getting various errors. Here's what I've tried, and here's what I've received:

evergreen=#
evergreen=# INSERT INTO biblio.record_entry (marc, last_xact_id) SELECT marc, 'IMPORT' FROM marcxml_import;
WARNING:  could not find ParserDetails.ini in /usr/local/share/perl/5.10.1/XML/SAX
CONTEXT:  compilation of PL/Perl function "maintain_control_numbers"
ERROR:  Unable to provide required features
CONTEXT:  PL/Perl function "maintain_control_numbers"
evergreen=#

I've been able to insert into various other test tables that I've set up, with no trouble. So, it's something about the biblio.record_entry table that does not like the column I'm trying to insert. However, trying to import anything else, results in the same errors.

I tried (using phppgadmin) creating a table like biblio.record_entry. I kept all constraints, defaults, and indexes and called it biblio.record_entry_entry. I was able to insert my data from marcxml_import successfully. However, when trying to insert from that to the main table, the same errors happen:

evergreen=#
evergreen=# INSERT INTO biblio.record_entry_entry (marc, last_xact_id) SELECT marc, 'IMPORT' FROM marcxml_import;
INSERT 0 23404
evergreen=# INSERT INTO biblio.record_entry SELECT * FROM biblio.record_entry_entry;
ERROR:  Unable to provide required features
CONTEXT:  PL/Perl function "maintain_control_numbers"
evergreen=#

Lastly, I've tried adding a test column to the biblio.record_entry table. Trying to insert into that, gives me a warning about marcxml that is not well formed

evergreen=#
evergreen=# INSERT INTO biblio.record_entry ( test ) SELECT marc FROM marcxml_import;
ERROR:  Attempted to INSERT MARCXML that is not well formed
evergreen=#

All that to say, I'm totally unsure of how to move forward. Do these errors make sense to anyone else? Is there any way around what I'm getting here? Is there a way, other than INSERT INTO, that I might be able to try to get this data into the appropriate table? Is it at all possible (let alone a good idea) to tell Evergreen that I want to use biblio.record_entry_entry, rather than biblio.record_entry?

Thanks for all your help!

Elliot J Voris


-----Original Message-----
From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Dan Scott
Sent: Thursday, December 15, 2011 09:51 PM
To: Evergreen Development Discussion List
Subject: Re: [OPEN-ILS-DEV] MARC Record Import from txt

On Thu, Dec 15, 2011 at 6:26 PM, Elliot Voris <evoris at slcconline.edu> wrote:
> Hi, Dev Group!
>
> I've made some great progress, albeit in a round-about way. For whatever reason, I could not get the .sql file to work properly. I ended up having to copy/paste in a few waves my Marc records into the marcxml_import table using webmin. Not ideal, but I now have all the records in my database.
>
> I'm having trouble inserting it into the biblio.record_entry table, now. I'm taking the line right out of the sample script you gave, but it's having the following effect:
>
> evergreen=#
> evergreen=# INSERT INTO biblio.record_entry (marc, last_xact_id) 
> SELECT marc, 'IMPORT' FROM marcxml_import;
> ERROR:  Unable to provide required features
> CONTEXT:  PL/Perl function "maintain_control_numbers"
> evergreen=#
>
> I found that there was mention in the changelog from 1.6.1.0 to 2.0.0 to the global flag 'cat.maintain_control_numbers', but I can't tell if I'm supposed to enable/disable it on my installation, or what. What do I need to do to insert my records from the one table to the next?

Elliot:

Rather than using Webmin, from the command line the command to load a random SQL file is:

psql -U <username> -h <hostname> -f <filename>

So... "psql -U evergreen -h localhost -f concerto.sql" should be all you need, possibly passing in a password as well.

I can't say I've ever seen "ERROR: Unable to provide required features" before... a quick Googling suggests that it's a problem with your Perl XML modules (http://cpansearch.perl.org/src/GRANTM/XML-SAX-0.99/SAX/ParserFactory.pm).
Assuming you've run the Makefile.install for Ubuntu Lucid on both OpenSRF and Evergreen, it should work, but at this point I'll have to defer to people with current Ubuntu experience.

Dan