[OPEN-ILS-GENERAL] importing 2000.000 records

[Marjolein Kremer]

Hi Jason,

Thanks for the indication.

Marjolein

-----Original Message-----
From: open-ils-general-bounces at list.georgialibraries.org [mailto:open-ils-general-bounces at list.georgialibraries.org] On Behalf Of Jason Stephenson
Sent: donderdag 24 maart 2011 16:43
To: open-ils-general at list.georgialibraries.org
Subject: Re: [OPEN-ILS-GENERAL] importing 2000.000 records

Quoting Marjolein Kremer <mkr at iisg.nl>:

> Hello,
>
> Does someone know what time it will take to import 2000.000 records  
> into Evergreen? We will be using EG 2.0.3.

Marjolein,

Unfortunately, no. The time depends on a number of factors. The most  
important of these are the power of your database server and how it is  
configured. The second is how you are loading the records and whether  
or not you load them in serial or parallel fashion.

I can tell you that it took 8 hours, 17 minutes, and 20 seconds to  
load 927,286 bibliographic records into our development database  
server with 24GB of RAM, 2 quad-core 2.5GHz CPUs, and PostgreSQL  
optimized to use the server's resources. That time was achieved with 7  
parallel load processes each processing 10,000 records simultaneously.  
We also loaded 508,003 authorities at the same time. (The authorities  
only took about half an hour to load, because Evergreen does much less  
server side processing with authority records.)

The above was done on a trunk installation, so it is not 2.0.3, but  
something more recent that requires PostgreSQL 9.0.

Hopefully, that gives you some idea of how long your load might take.

HTH,
Jason Stephenson
Merrimack Valley Library Consortium




>
> Marjolein Kremer
> IISG
> Books and Serials Acquisitions
>
>
>
>
>
>