[OPEN-ILS-GENERAL] Importing records from Unicorn

[Warren Layton]

On Tue, Oct 7, 2008 at 12:30 AM, Dan Scott <denials at gmail.com> wrote:
> 2008/10/6 Warren Layton <warren.layton at gmail.com>:
>> * YAZ version 2.1.56
>> * MARC::Record is from the Ubuntu package "libmarc-record-perl", version 2.0.0-1
>> * marc2bre I'm less sure. I got it from the import demo on the Evergreen site
>> (http://open-ils.org/~denials/import_demo.tar.gz)
>
> Hmm - that could be a problem. Assuming you're running against
> Evergreen trunk, you will probably want to use the versions of
> marc2bre.pl, parallel_pg_loader.pl, direct_ingest.pl and
> quick_metarecord_map.sql from trunk. I ran the import_demo against
> 1.2.3.1 last week and noticed that the namespaces in the generated XML
> didn't quite look right in comparison to records loaded via Vandelay.
> import_demo was last updated seven months ago, so it probably needs an
> update to sync up with what Evergreen expects today.
>
> You can also upgrade yaz to 3.0.36; that's what is used in the most
> current version of Makefile.install.

Update: I upgraded yaz and grabbed the latest marc2bre.pl and this
seems to work for the first steps.

Once I get to the execution of parallel_pg_loader.pl, I see a few
warnings like this:
   "Argument "\\N" isn't numeric in int at parallel_pg_loader.pl line
110, <> line 2263351."

A more serious problem occurs when the loader finishes and I try to do
a "\i pg_loader-output.sql" on the psql prompt:

evergreen=# \i /home/opensrf/import_demo/pg_loader-output.sql
SET
BEGIN
psql:/home/opensrf/import_demo/pg_loader-output.sql:5: ERROR:
duplicate key violates unique constraint "record_entry_pkey"
CONTEXT:  COPY record_entry, line 186269: "t    now     1       f
 now     1       canadianentomologistentomological       10010488
  IMPORT-1223388363.688   <record    xmlns:xs..."
psql:/home/opensrf/import_demo/pg_loader-output.sql:6: ERROR:  current
transaction is aborted, commands ignored until end of transaction
block
psql:/home/opensrf/import_demo/pg_loader-output.sql:7: ERROR:  current
transaction is aborted, commands ignored until end of transaction
block
psql:/home/opensrf/import_demo/pg_loader-output.sql:8: ERROR:  current
transaction is aborted, commands ignored until end of transaction
block
psql:/home/opensrf/import_demo/pg_loader-output.sql:9: ERROR:  current
transaction is aborted, commands ignored until end of transaction
block
psql:/home/opensrf/import_demo/pg_loader-output.sql:10: ERROR:
current transaction is aborted, commands ignored until end of
transaction block
psql:/home/opensrf/import_demo/pg_loader-output.sql:12: ERROR:
current transaction is aborted, commands ignored until end of
transaction block
psql:/home/opensrf/import_demo/pg_loader-output.sql:13: ERROR:
current transaction is aborted, commands ignored until end of
transaction block
psql:/home/opensrf/import_demo/pg_loader-output.sql:14: ERROR:
current transaction is aborted, commands ignored until end of
transaction block
psql:/home/opensrf/import_demo/pg_loader-output.sql:15: ERROR:
current transaction is aborted, commands ignored until end of
transaction block
evergreen=#


When running marc2bre.pl, I used the option "--idfield=001". My guess
was that the first error in the above block indicates that there are
two identical 001 fields among the records. Is this the correct
interpretation of this error?

Thanks!
  Warren