[OPEN-ILS-DEV] MARC Record Import from txt

[Elliot Voris]

Thank you so much, Dan!

This has been really helpful. The sample script you gave me worked without a flaw! I did run into one problem so far, though. I uploaded my file (which was largely based on yours) and ran it through webmin (Is there a preferred way to run a .sql script on Evergreen, or does it even make a difference?), and it didn't give me any output until the script was completely run (about 8 hourse, btw). The output that I finally got was a list of the following:

**Note, the carat (^) should be placed under the first single-quote (')**

BEGIN
CREATE TABLE
psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 8: ('<record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instan...
         ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 9: ('<record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instan...
         ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 10: ('<record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instan...
          ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 11: ('<record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instan...

There's a line like that for each entry in the file. And then there's this at the bottom of the output:

psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 23403: ('<record xmlns:xsi="http://www.w3.org/2001/XMLSchema-instan...
             ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
psql:/tmp/.webmin/318880_1_exec_file.cgi:23410: WARNING:  nonstandard use of escape in a string literal
LINE 23404: ('<record xmlns="http://www.loc.gov/MARC21/slim" xmlns:marc=...
             ^
HINT:  Use the escape string syntax for escapes, e.g., E'\r\n'.
INSERT 0 23404
psql:/tmp/.webmin/318880_1_exec_file.cgi:23412: ERROR:  Tag "" is not a valid tag. at /usr/local/share/perl/5.10.1/MARC/File/SAX.pm line 113
CONTEXT:  PL/Perl function "maintain_control_numbers"
psql:/tmp/.webmin/318880_1_exec_file.cgi:23414: ERROR:  current transaction is aborted, commands ignored until end of transaction block
ROLLBACK

 I mimicked the formatting and syntax of the file you sent very closely. A MARC record in my script looks like this: ('<MARC record info>'), so I'm not sure where I went wrong. I find it interesting that the first error appears to be on line 8, but lines 6,7 have MARC records on them, as well.

Any idea how I might be able to get this info into my database?

Thanks, again!

Elliot J Voris

-----Original Message-----
From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Yamil Suarez
Sent: Tuesday, December 13, 2011 08:49 AM
To: Evergreen Development Discussion List
Subject: Re: [OPEN-ILS-DEV] MARC Record Import from txt

Dan (or others),

I have a documentation related question for you based on your response  
to the original email question. Currently the 1.6 and 2.0 official  
documentation make reference to the problem of "namespace prefixes."  
Should that be removed from the 1.6 and 2.x version, or just the 2.x?

Here are direct links to the pertinent sections...

http://docs.evergreen-ils.org/2.0/draft/html/migratingdata.html#migratingbibrecords
http://docs.evergreen-ils.org/1.6/draft/html/migratingdata.html#migratingbibrecords

Thanks in advance,
Yamil




On Dec 11, 2011, at 6:55 PM, Dan Scott wrote:

> Elliot:
>
> On Fri, Dec 9, 2011 at 5:20 PM, Elliot Voris <evoris at slcconline.edu>  
> wrote:
>> Thanks, John!
>>
>> Thanks for the point in the right direction! Unfortunately, my  
>> records do
>> have namespace identifiers. The wiki says I need to "do a horrible  
>> thing and
>> ensure that there are no namespace prefixes in front of the element  
>> names."
>> How might I do that? I played a bit with find/replace, but I don't  
>> know what
>> namespaces I might be looking for.
>
> You probably don't need to worry about namespaces these days, I wrote
> that wiki entry back in the Evergreen 1.2 days or earlier.
>
> As an FYI, I submitted a sample file a few days back that imports 100
> MARCXML records and then generates call numbers and copies for the
> records. Although it was intended to provide a standard set of records
> for the purposes of testing and documentation, you might want to
> borrow from this example for your own import efforts:
>
> http://git.evergreen-ils.org/?p=working/Evergreen.git;a=blob;f=Open-ILS/tests/datasets/concerto.sql;h=0dec4ed9051382e49740f67e623e4cfb11076599;hb=db2862d3be35fa0cc33c318cf5568b9de48c0caf