[OPEN-ILS-DEV] distributed perl?

[Mike Rylander]

On Feb 7, 2008 10:20 AM, John Fink <john.fink at gmail.com> wrote:
> This isn't really an Evergreen specific question, but it does involve
> Evergreen, so.
>
> I'm in the midst of processing a giant pack of bibs (I think about two and a
> half million) via the marc2bre/direct_ingest/pg_loader tango delineated
> here:
> http://open-ils.org/dokuwiki/doku.php?id=evergreen-admin:importing:bibrecords&s=gutenberg
> .  This is, as you might expect from an old desktop, taking Forever and a
> Day.  Is there a way to parallelize perl like I've seen for C?  Or is it
> just a better idea to use beefy machines for the heavy lifting?

If your old desktop has more than one CPU, then you could pipe (or
tee, if you want to keep the bre output around) into the direct ingest
process, and pipe (or tee) that into the parallel_pg_loader process.

--miker

>
> jf
>
>
>
> --
> http://libgrunt.blogspot.com -- library culture and technology.



-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com