[OPEN-ILS-DEV] Re: pg_loader getting killed

[John Fink]

Oh man, I love you guys.  Thanks!  I'll give it a spin and report.

jf

On Feb 20, 2008 11:15 AM, Mike Rylander <mrylander at gmail.com> wrote:

> On Feb 20, 2008 10:26 AM, John Fink <john.fink at gmail.com> wrote:
> > Oh, and this is Evergreen 1.2.1.0, sorry for not stating that earlier.
> >
>
> Grab parallel_pg_loader.pl from trunk* and try that.  It avoids
> loading the entire file into memory before writing the files, and was
> built for just such a situation.  It works fine with 1.2.x
>
> *
> http://svn.open-ils.org/trac/ILS/browser/trunk/Open-ILS/src/extras/import/parallel_pg_loader.pl
>
> --miker
>
> > jf
> >
> >
> >
> > On Feb 20, 2008 9:18 AM, John Fink <john.fink at gmail.com> wrote:
> >
> > > Hey folks,
> > >
> > > I've tried a couple of times unsuccessfully to run pg_loader on what
> is
> > admittedly a vastly huge (ls shows me filesize 17077683247) ingest file.
> > For a while it hums merrily along, but then I inevitably get something
> like
> > the following:
> > >
> > > 8643000 363.398018751032Killed
> > >
> > > Complete with a zero-length sql.  Am I simply biting off more than I
> can
> > chew?  Is it possible to easily split the ingest file for easier
> loading?
> > It looks like to me that pg_loader is trying to load everything into
> memory
> > before writing the sql (hence the zero length).
> > >
> > > jf
> > >
> > >
> > >
> > > --
> > > http://libgrunt.blogspot.com -- library culture and technology.
> >
> >
> >
> > --
> > http://libgrunt.blogspot.com -- library culture and technology.
>
>
>
> --
> Mike Rylander
>  | VP, Research and Design
>  | Equinox Software, Inc. / The Evergreen Experts
>  | phone:  1-877-OPEN-ILS (673-6457)
>  | email:  miker at esilibrary.com
>  | web:  http://www.esilibrary.com
>



-- 
http://libgrunt.blogspot.com -- library culture and technology.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://list.georgialibraries.org/pipermail/open-ils-dev/attachments/20080220/5e2ee50f/attachment.html