[OPEN-ILS-GENERAL] Import issues

Thu Aug 13 22:53:09 EDT 2009

Hi Dan,

Thank you for your prompt response.

Please find the attachment herewith. The attachment includes outputs of
three MARC records containing 250-300 bibliographic information.

Thank you.

With kind regards,
Dibyendra

On Thu, Aug 13, 2009 at 11:11 PM, Dan Scott <denials at gmail.com> wrote:

> Hi Dibyendra:
>
> Can you please attach some zipped output (BRE, ingest, and SQL) files
> as well as the errors to help us work out where the duplication is
> occurring?
>
> Dan
>
> 2009/8/13 Dibyendra Hyoju <dibyendra at gmail.com>:
> > Hello Dan,
> >
> > Thanks for your prompt response. I tried those options with marc2bre.pl
> and
> > now the 'marc2bre.pl' produced no error. But, the problem about
> duplication
> > still persists while executing the SQL. I rebuilt the database several
> > times, and tried 'marc2bre.pl' with all the suggested options like you
> have
> > suggested.
> >
> > I performed the following commands during the import process and they
> > produced SQL for each MARC records:
> >
> > #perl marc2bre.pl --db_user postgres --db_host localhost --db_pw
> evergreen
> > --db_name evergreen --encoding UTF8 --idfield 852 --idsubfield x
> --tcnfield
> > 852 --tcnsubfield x 8500.mrc > 8500.bre
> > #perl direct_ingest.pl ~/8500.bre > ~/8500.ingest
> > #perl pg_loader.pl -or bre -or mrd -or mfr -or mtfe -or mafe -or msfe -or
> > mkfe -or msefe -a mrd -a mfr -a mtfe -a mafe -a msfe -a mkfe -a msefe
> > --output=8500.sql < ~/8500.ingest
> > #cp 8500.sql ~
> > #psql -U evergreen evergreen
> > evergreen=# \i ~/8500.sql
> >
> > After rebuilding the database, any generated SQL is executed
> successfully.
> > But, after the first successful execution of any generated SQL, any other
> > generated SQL files are not executed. I've attached the generated SQL
> errors
> > herewith.
> >
> > I look forward to hearing from you.
> >
> > Thank you.
> >
> > --
> > Dibyendra Hyoju
> > Madan Puraskar Pustakalaya
> > Patan Dhoka, Lalitpur
> > Nepal
> >
> > On Thu, Aug 13, 2009 at 12:14 PM, Dan Scott <denials at gmail.com> wrote:
> >>
> >> Sorry Dibyendra, those options should have been --tcnfield and
> >> --tcnsubfield
> >>
> >> Try those, they should work in 1.4.
> >>
> >> Dan
> >>
> >> 2009/8/13 Dibyendra Hyoju <dibyendra at gmail.com>:
> >> > Hello Dan,
> >> >
> >> > Thank you very much for your response.
> >> >
> >> > I used the options: 'tcn_field' and 'tcn_subfield', but they are not
> >> > recognized by the marc2bre.pl. I executed the marc2bre.pl in following
> >> >  way and the output is as follows:
> >> >
> >> >
> >> >
> dibyendra-laptop:/home/opensrf/Evergreen-ILS-1.4.0.4/Open-ILS/src/extras/import#
> >> > perl marc2bre.pl --db_user postgres --db_host localhost --db_pw
> >> > evergreen --db_name evergreen --encoding UTF8 --idfield 852
> >> > --idsubfield x --tcn_field 852 --tcn_subfield x 8500.mrc > 8500.bre
> >> > Unknown option: tcn_field
> >> > Unknown option: tcn_subfield
> >> >
> >> > I guess, these options: 'tcn_field' and 'tcn_subfield' are not
> >> > implemented on EG 1.4 yet. Is that so? I couldn't find EG 1.6 on the
> >> > download page. We're planning to migrate our 10,000 validated MARC
> >> > records into Evergreen.
> >> >
> >> > Thank you.
> >> >
> >> > --
> >> > Dibyendra Hyoju
> >> > Madan Puraskar Pustakalaya
> >> > Patan Dhoka, Lalipur
> >> > Nepal
> >> >
> >> > On Tue, Aug 11, 2009 at 10:15 AM, Dan Scott <denials at gmail.com>
> wrote:
> >> >>
> >> >> 2009/8/8 Dibyendra Hyoju <dibyendra at gmail.com>:
> >> >> > Hello all,
> >> >> > I imported the sample records again on the another machine having
> >> >> > Evergreen
> >> >> > 1.4, and after importing that record, I couldn't import any records
> >> >> > from
> >> >> > other MARC files. I have attached the record '8500.mrc' and
> >> >> > '8400.mrc'
> >> >> > herewith. I first executed the SQL generated from 8500.mrc'
> >> >> > successfully.
> >> >> > Then, I couldn't execute the SQL generated from '8400.mrc'. The
> error
> >> >> > is
> >> >> > same as before like 'ERROR:  duplicate key violates unique
> constraint
> >> >> > "biblio_record_unique_tcn"'. I only used the option "--encoding
> UTF8"
> >> >> > this
> >> >> > time. I tried few other records, but I got the same error. Few of
> the
> >> >> > tested
> >> >> > records are attached herewith if somebody wants to volunteer the
> >> >> > test. If
> >> >> > anyone has faced this problem before and have found the solution,
> >> >> > please
> >> >> > help. Any help will be highly appreciated.
> >> >>
> >> >> Sorry for the delayed reply, I'm on leave at the moment and not
> >> >> connected very often.
> >> >>
> >> >> marc2bre.pl doesn't really deal with automatically generated TCNs all
> >> >> that well, so you're best off explicitly identifying a source for the
> >> >> TCN. To avoid getting duplicate TCN values and record ID values if
> all
> >> >> of your records follow the same pattern with the 852 $x field /
> >> >> subfield accession number identifier, you should also use the
> >> >> --tcn_field / --tcn_subfield options for marc2bre.pl:
> >> >>
> >> >> perl marc2bre.pl --db_user postgres --db_host localhost --db_pw
> >> >> evergreen --db_name evergreen --encoding UTF8 --idfield 852
> >> >> --idsubfield x --tcn_field 852 --tcn_subfield x 8400.mrc > 8400.bre
> >> >>
> >> >> I tried importing both your 8000.mrc and 8400.mrc files with the and
> >> >> these options worked fine for me with no duplicate value warnings.
> >> >> This is with Evergreen rel_1_6_0 on Ubuntu 8.04, but marc2bre.pl (the
> >> >> most critical script for parsing out TCN and record number) shows no
> >> >> significant differences between 1.4 and rel_1_6_0.
> >> >>
> >> >> --
> >> >> Dan Scott
> >> >> Laurentian University
> >> >
> >>
> >>
> >>
> >> --
> >> Dan Scott
> >> Laurentian University
> >
>
>
>
> --
> Dan Scott
> Laurentian University
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://libmail.georgialibraries.org/pipermail/open-ils-general/attachments/20090814/829ae286/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: outputs.tar.gz
Type: application/x-gzip
Size: 445264 bytes
Desc: not available
Url : http://libmail.georgialibraries.org/pipermail/open-ils-general/attachments/20090814/829ae286/attachment-0001.bin 