[OPEN-ILS-DEV] bibload errors

[Dan Scott]

On 24/01/2008, Grant Johnson <FGJohnson at upei.ca> wrote:
> Exported 25,000 SIRSI records.
> Seems to have gone through marc2bre, direct_ingest.pl and pg_loader OK.
> Files created that look normal.
>
> But   #  \i newrecords.sql results in:
>
> ...psql:books.sql:768112: invalid command \N
> psql:books.sql:768113: invalid command \N
> psql:books.sql:768136: invalid command \N
> psql:books.sql:768137: invalid command \N
> psql:books.sql:768357: invalid command \N
> psql:books.sql:768358: invalid command \N
> psql:books.sql:768378: invalid command \N
> psql:books.sql:768379: invalid command \N
> psql:books.sql:768403: invalid command \N
> psql:books.sql:768404: invalid command \N
> psql:books.sql:871957: invalid command \.
> psql:books.sql:878612: ERROR:  syntax error at or near "2" at character 1
> psql:books.sql:878612: LINE 1: 2 a 020 086196182x
> psql:books.sql:878612:         ^
>
> What am I missing? All records are monographs.
> Cheers

Hi Grant:

Is the database clean? It would be interesting to see the very start
of the errors; I'm wondering whether there might be a duplicate ID.

Let's say your bibs were processed assuming a starting point of ID = 1
(the default with marc2bre.pl), but you already had a bib record in
the database (say, if you were trying out z39.50 import as a test),
then the first record that you tried to insert would fail - and maybe
that would cause a cascade of errors if that rolls back the entire
COPY transaction for that table, thereby preventing the foreign keys
from the subsequent tables from being able to resolve. (I'm a little
weak on my understanding of COPY's behaviour on error, so I could be
out in left field here).

>From the PostgreSQL docs
(http://www.postgresql.org/docs/8.2/static/sql-copy.html):

"COPY stops operation at the first error. This should not lead to
problems in the event of a COPY TO, but the target table will already
have received earlier rows in a COPY FROM. These rows will not be
visible or accessible, but they still occupy disk space. This may
amount to a considerable amount of wasted disk space if the failure
happened well into a large copy operation. You may wish to invoke
VACUUM to recover the wasted space."

-- 
Dan Scott
Laurentian University