[OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading bibs.

[Frances Dean McNamara]

I believe NONE of the records loaded got rows.  Perhaps trying the bib_ids.list for ranges of records to see if it works.  You say that would be a really long thing?  Does that do more thand the metabib.rec_descriptor inserts?

Or we could use your first suggestion.  We actually do have the original input files broken down to about 59 files I think.  I will talk to Dale when he is in.  If we figure out which way to do it we can probably script it.

Thanks.

Frances

-----Original Message-----
From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Mike Rylander
Sent: Friday, June 12, 2009 9:03 AM
To: Evergreen Development Discussion List
Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading bibs.

On Fri, Jun 12, 2009 at 9:47 AM, Frances Dean
McNamara<fdmcnama at uchicago.edu> wrote:
> Ah, I see, it is pulling out stuff like type of date, date1, date2 from the fixed field, bib lvl, enc lvl, looks like mainly 008 type of data?  Is there a script we could run for a while to build this?  We have over 5 million bibs.  Thanks.  Frances
>

You can use the stock ingest scripts, and avoid intermediate files by
doing something like:

$ marc2bre.pl --marctype=xml [... other options] | direct_ingest.pl |
grep '"__c":"mrd"' > mrd.ingest

for 1.4.  If you're working with 1.2, you'll need to change the grep,
but that will get you the mrd data.

Another other option is to get a list of all the bibs that lack 'mrd's
in a file and re-ingest them.  In psql:

  \t
  \o /tmp/bib_ids.list
  SELECT b.id FROM biblio.record_entry b LEFT JOIN
metabib.rec_descriptor d ON (b.id = d.record) WHERE r.id IS NULL b.id
> 0 AND NOT b.deleted;
  \o
  \t

Then as the opensrf user, at the command line:

$ for i in `cat /tmp/bib_ids.list`; do echo "request open-ils.ingest
open-ils.ingest.full.biblio.record $i"; done | srfsh

That will take, um, a really long time to complete.  If anything
happens to stop that, just start over with a fresh bib_ids.list file
-- the SQL above will give you only te records that are missing an
'mrd'.

--miker

> -----Original Message-----
> From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Frances Dean McNamara
> Sent: Friday, June 12, 2009 8:37 AM
> To: Evergreen Development Discussion List
> Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading bibs.
>
> Dale has not replied to this yet, but I don't think we have kept the interim files due to file space (big database).
>
> Are their scripts needed to regenerate that table from what is in the db now?  What is in that table anyhow?
>
> Frances McNamara
> University of Chicago
>
> -----Original Message-----
> From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Mike Rylander
> Sent: Friday, June 12, 2009 7:12 AM
> To: Evergreen Development Discussion List
> Subject: Re: [OPEN-ILS-DEV] No rows in metabib.rec_descriptor after loading bibs.
>
> On Thu, Jun 11, 2009 at 7:23 PM, Dale Arntson<arnt at uchicago.edu> wrote:
>> Hi All,
>>
>> After loading our bib records into evergreen, I got zero hits on searches in
>> the evergreen client. I traced the problem back to the fact that there are
>> no records in the metabib.rec_descriptor table. The other metabib tables
>> seem fully populated. Here are the flags I used in parallel_pg_loader.
>>
>> perl parallel_pg_loader.pl -order bre -order mrd -order mfr -order mtfe
>>  -order mafe -order msfe -order mkfe -order msefe -autoprimary mrd
>> -autoprimary mfr -autoprimary mtfe -autoprimary mafe -autoprimary msfe
>> -autoprimary mkfe -autoprimary msefe
>>
>> Any ideas what I did wrong? Any ideas how to fix it? The bibs took a long
>> time to load. I would rather not redo it, if I don't have to.
>
> If you have the intermediate bib processing files, particularly the
> output of direct_ingest, make sure that file contains 'mrd' rows.  For
> OpenSRF 0.9 (Evergreen 1.2.x), those will start with '/*--S mrd--*/',
> and in OpenSRF 1.0.x they will start with '[{"__c":"mrd","__p"'.
>
> If you can find that file and it contains 'mrd' lines, then we can
> simply regenerate that table's worth of data. Otherwise you'll need to
> reprocess the bib records in order to get that data, but you won't
> need to reload the entire dataset.
>
> --
> Mike Rylander
>  | VP, Research and Design
>  | Equinox Software, Inc. / The Evergreen Experts
>  | phone:  1-877-OPEN-ILS (673-6457)
>  | email:  miker at esilibrary.com
>  | web:  http://www.esilibrary.com
>



-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com