[OPEN-ILS-DEV] Bulk importing copies into Evergreen-newbie question

[Nella Lall]

Dan and others,

Finally, had a little time to try more on this today and despite various 
attempts, I just am not getting any copies to load into my evergreen 
database via the import_holdings.pl script.  I have sprinkled debug 
statements throughout the script and even though it seems to properly 
know where my marcfile is, the @records array seems empty and the script 
never enters the foreach record loop.  Ideas?

My perl is weak so I don't understand everything the script is doing yet 
(obviously)!

The setup we have for our evergreen is the default installed one so that 
our library names simply are: Example Consortium, Example System 1, 
Example Branch 1 etc etc.

Here is the output from the script with all my print statements:

sudo perl import_holdings.pl --marc_file ~/test1g.xml
beginning the import_holdings script
my holding_tag is: ./datafield[@tag="999"]
my bc is 999 subfield i: i
my lbl is 999 subfield a: a
my own is 999 subfield m: m
my pr is 999 subfield p: p
my cpn is 999 subfield c: c
my avail is 999 subfield k: k
my marc_file: /home/evergreen-admin/test1g.xml
my item_barcode: i
my lbl is 999 subfield a (item_call_number): a
my own is 999 subfield m (owning_lib): m
my pr is 999 subfield p (price): p
my cpn is 999 subfield c (copy_number): c
my avail is 999 subfield k (copy status): k
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my lib is: ARRAY(0x8501e70)
my tcn_sth is: DBI::st=HASH(0x8418f90)
my xact_id is: 1203537392.26222
my doc is: XML::LibXML::Document=SCALAR(0x8502164)
my xc is: XML::LibXML::XPathContext=SCALAR(0x8502194)
I have -1 in the record array

##############
So, it ends and never goes into the foreach record loop (snippet below) 
where I would expect it to find the information it needs from my marcfile??

foreach my $record (@records) {

        print "in the foreach loop of records\n";
        my $tcn;
 ...

Can anyone spot my problem that I just can't lay a finger on? 

Thanks,
Nella Lall


Dan Scott wrote:
> On 20/11/2007, Nella Lall <ndlall at ucalgary.ca> wrote:
>   
>> I am completely new to Evergreen and after successfully loading one
>> lonely marc record into our evergreen database,
>> I got stumped when I tried to load a marc record's two related copies.
>>
>> Here is what I have done based on the Evergreen admin documentation
>> found at:
>> http://open-ils.org/dokuwiki/doku.php?id=evergreen-admin:importing:bibrecords#adding_copies_to_bibliographic_records_in_evergreen
>>
>>
>> I began the process of taking that ONE marc record (with the in tag 999)
>> and converted it to MARC21XML with MarcEdit.  I transferred that
>> converted file up to the evergreen server.  I called my file
>> test1b.xml.  Here is a snippet from my
>> MARC21XML file show the two 999 tags:
>>
>> <datafield tag="999" ind1=" " ind2=" ">
>> <subfield code="a">WA590 .C365 1998</subfield>
>> <subfield code="w">NLM</subfield>
>> <subfield code="c">1</subfield>
>> <subfield code="i">00009728368</subfield>
>> <subfield code="d">11/14/2007</subfield>
>> <subfield code="e">11/14/2007</subfield>
>> <subfield code="k">CHECKEDOUT</subfield>
>> <subfield code="l">RGH</subfield>
>> <subfield code="m">BR1</subfield>
>> <subfield code="n">1</subfield>
>> <subfield code="r">M</subfield>
>> <subfield code="s">Y</subfield>
>> <subfield code="t">2WEEK</subfield>
>> <subfield code="u">5/15/2007</subfield>
>> <subfield code="x">UNKNOWN</subfield>
>> <subfield code="z">RGH</subfield>
>> </datafield>
>>
>> <datafield tag="999" ind1=" " ind2=" ">
>> <subfield code="a">WA590 .C365 1998</subfield>
>> <subfield code="w">NLM</subfield>
>> <subfield code="c">2</subfield>
>> <subfield code="i">00009714384</subfield>
>> <subfield code="k">WHKC-REF</subfield>
>> <subfield code="l">WHKC-REF</subfield>
>> <subfield code="m">BR1</subfield>
>> <subfield code="r">Y</subfield>
>> <subfield code="s">Y</subfield>
>> <subfield code="t">LIBONLY</subfield>
>> <subfield code="u">8/14/2007</subfield>
>> <subfield code="x">UNKNOWN</subfield>
>> <subfield code="z">WHKC</subfield>
>> </datafield>
>>
>> </record>
>> </collection>
>>
>>
>> I grabbed the import_holdings.pl file off of
>> http://svn.open-ils.org/trac/ILS/browser/trunk/Evergreen/src/extras/import/import_holdings.pl
>>
>> and changed the settings in line 45 my
>> ($db_driver,$db_host,$db_name,$db_user,$db_pw) =
>> to what we have setup for our evergreen username, password etc etc.
>>
>> Then I ran:
>>
>> perl import_holdings.pl --marc_file test1b.xml
>>
>>
>> There were no errors and it created the 2 expected files in my working
>> directory:
>> asset_volume.sql and asset_copy.sql
>>
>> But both are "blank inside" without the specific data from my input
>> file's xml 999 subfields.
>>
>> eg. content of my asset_copy.sql is:
>>
>> SET CLIENT_ENCODING TO 'UNICODE';
>> COPY asset.copy
>> (id,circ_lib,editor,creator,barcode,call_number,copy_number,status,loan_duration,fine_level,circulate,deposit,deposi
>>
>> t_amount,price,ref,opac_visible) FROM STDIN;
>> \.
>> SELECT setval('asset.copy_id_seq'::TEXT, 1);
>>
>>
>> I did continue onto the next step of doing:
>> Import the volume and copy SQL files into your Evergreen database:
>>
>> psql evergreen
>> # \i asset_volume.sql
>> # \i asset_copy.sql
>>
>>
>> Again with no error messages but neither copy has been attached to the
>> bib.record in the evergreen database.
>>
>> So, can anyone out there see my error or give me clues on how to
>> successfully import copies??  Thanks so much,
>>     
>
> Hi Nella:
>
> Welcome to Evergreen!
>
> Have you created the library in Evergreen that corresponds to the
> owning library field ( <subfield code="m">BR1</subfield> -- so BR1)?
> You can use the cgi-bin/config.cgi to add, remove, or change libraries
> (often called "org units" in EG).
>
> Also, the "k" subfield (which resolves to WHKC-REF in your examples)
> should map to a known circulation status as commented in the script:
>
> #  The keys of this hash should be the string values stored in your legacy
> #  system that map to the copy statuses in Open-ILS.  If you don't see a
> #  legacy status here that you need to carry over to your new Open-ILS install
> #  you can use the "Copy Statuses" bootstrapping CGI to create an entry for it.
> #  Then simply a key for the legacy status that points to the SysID of the new
> #  Open-ILS Copy Status.
>
> Despite what Don suggested, I don't think the lack of a price subfield
> will cause any problems, as the script automatically sets the price to
> $0.00 if it doesn't find a value. The script bails silently on a given
> holding if it can't find a barcode, owning library, or label (call
> number).
>
> Perhaps you could try sprinkling some debug print statements
> throughout the script to see how far it goes? I used the script to
> import holdings for a test run of 40,000 items from a Unicorn export
> of records, and it seemed to work well enough for me.
>
>