[OPEN-ILS-GENERAL] Import issues

[Dan Scott]

2009/8/26 Dibyendra Hyoju <dibyendra at gmail.com>:
> Hi Dan,
> I couldn't solve the problem about the duplication error " ERROR:  duplicate
> key violates unique constraint "biblio_record_unique_tcn", even after using
> the option "--used_tcn_file". The file that is used by the option contains
> two lines like you had suggested. I searched for the solution for couple of
> days related to this issue, but couldn't find the relevant solutions. So,
> would you please kindly send me the file required for the
> option "--used_tcn_file", or suggest me some alternative solution by which I
> can import our library records into Evergreen? We are currently testing the
> features of Evergreen 1.4.
> I look forward to hearing from you.

Hi Dibyendra:

I just imported both 8400.mrc and 8500.mrc successfully into an
Evergreen 1.4.0.4 system.

I suspect the problem that you're running into at the moment is that
you're processing 8400.mrc and 8500.mrc first, then loading their SQL
files into the database.

This approach will result in duplicate TCN values and failure because
marc2bre.pl is generating TCN values based on the record ID, which is
just a numeric sequence in the database. When you run marc2bre.pl, it
grabs the current maximum record ID from the database and uses that as
the starting point for the record IDs & TCN values that it generates.
So if you process both files first, then load them into the database,
they will both have been based on the same starting point for record
ID.

On the other hand, if you process 8400.mrc first and load 8400.sql
into the database, then when you process 8500.mrc marc2bre.pl will
start generating record IDs that are beyond the range that were
generated for 8400.mrc. So, your safest approach would be to process
each file and load it into the database.

Alternately, if you want to process a bunch of files in batch, and you
know you have 100 records per file, you could use the "--startid"
parameter to marc2bre.pl to force it to start at a new, unused range
of record IDs for each batch. For example:

perl marc2bre.pl --startid 8401 ... 8400.mrc > 8400.bre
...
perl marc2bre.pl --startid 8501 ... 8400.mrc > 8500.bre
...

Sorry for the delays in responding, I've been on leave and not
spending much time online.

(As an aside, marc2bre.pl could probably stand to be taught to be
smarter about parsing & generating TCN values, but this should get you
by for now).

-- 
Dan Scott
Laurentian University