[OPEN-ILS-DEV] Unexpectedly dying of authority_control_fields script when parallel processing

[Chris Roosendaal]

Dear all,

I would like to bring up this subject again because we will try to run the
authority_control_fields.pl script on our database another time this week.
We divided the whole process in eight batches, and since every batch takes
between 8 and 10 hours to complete, we would definitely like a parallel
approach. Unfortunately this approach did not succeed earlier as can be read
in this email thread.

I would like to know: do the problems we encounter have to do with our
Evergreen setup? We have a machine to run the scripts on, which is a
relatively slow dual core machine, connected to a separate fast machine with
eight cores. Could the scripts run into trouble because the CPU is too slow
to run 8 authority_control_fields.pl processes in parallel? Or maybe a lack
of memory?

Thanks in advance,

Chris Roosendaal.


> The authority_control_fields script *is* running against the same database
> server (after all: that is our Evergreen database)
>
> but the problems Chris is reporting [see the May 16th posting] happened
> days
> before we ran into those other problems Daniël discussed with you on this
> list
> [1].
>
> Sooo: any thoughts?
>
> Repke
>
> [1] not enough disk space for Evergreen's PostgreSQL while importing MARC
> data -
> which ruined the database to the extent that Evergreen couldn't connect to
> it
> anymore
>
> Op 17 mei 2011, om 22:55 heeft Dan Scott het volgende geschreven:
>
>> On Tue, May 17, 2011 at 09:40:46PM +0200, Repke de Vries wrote:
>>
>>> Posting on behalf of Chris at IISH, Amsterdam (his dev-list message
>>> doesn't get
>>> through)
>>>
>> I'm not a list administrator, so I don't know if Chris is subscribed to
>> the list from that email address.
>>
>> Is the script by any chance running against the same database server that
>> Daniël Kollmer reported was dying when open-ils.cstore was attempting to
>> connect to it? If so, all bets are off.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20110526/b2a988fc/attachment.htm