[OPEN-ILS-DEV] Conversion conundrums

[Don Hamilton]

Hi again.
 
Today, I'd like some advice/suggestions for loading huge gobs of marc
records.
 
In an attempt to use both halves of my dual cpu server, and do it all
at once, I tried 
 
"marc2bre | direct_ingest.pl |  pg_loader.pl > sql"...
 
Two things are not right with this approach...
 
1) the three idlers overwrite each other
2) after about 3 hours of merrily churning away, I get "out of memory",
and the system has a schizophrenic swapping party until I kill the three
tasks.
 
I think I'd like to have marc2bre read two 'extra' parameters,
startatwhat and howmanyshouldIdo, and well, start at the startat record
in the input file and do howmanyshouldido...
 
BUT..
 
Should I make them parameters? Or should they be read/written from a
control source somewhere so the above command line can re-iterate itself
over a large input file?
 
don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://list.georgialibraries.org/pipermail/open-ils-dev/attachments/20070606/d9989c07/attachment.html