[OPEN-ILS-GENERAL] ***SPAM*** Re: ***SPAM*** Importing bib and holdings data into Evergreen 2.4.2
Sharp, Chris
csharp at georgialibraries.org
Tue Oct 15 07:23:23 EDT 2013
Hi Vaclav,
6.6 GB sounds a bit tight to me and it appears that it's not enough to accommodate the import Linda is attempting. Any chance at increasing that size? I think it will be necessary based on the error Linda shared in her post.
Hope that's helpful,
Chris
----- Original Message -----
> From: "Václav Jansa" <vaclav.jansa at gmail.com>
> To: open-ils-general at list.georgialibraries.org
> Sent: Tuesday, October 15, 2013 6:16:27 AM
> Subject: [OPEN-ILS-GENERAL] ***SPAM*** Re: ***SPAM*** Importing bib and holdings data into Evergreen 2.4.2
>
> Hello Chris,
> i reply as a IT part of our Green home team;-)
>
> Its clean EG 2.4.2 installation on 64bit debian-wheezy, virtual
> machine
> use 10GB image
> free space is
> opensrf at evergree-nb:~/KKB$ df -h
> Filesystem Size Used
> Avail
> Use% Mounted on
> rootfs 9.6G 2.5G
> 6.6G
> 28% /
> udev 10M 0 10M
> 0% /dev
> tmpfs 202M 216K
> 202M 1% /run
> /dev/disk/by-uuid/e1cc81ee-2958-4d5d-941f-eab6798efe35 9.6G 2.5G
> 6.6G
> 28% /
> tmpfs 5.0M 0 5.0M
> 0% /run/lock
> tmpfs 497M 0 497M
> 0% /run/shm
>
> Old migration steps (from ESI migration tools) work for bibliographic
> records almost fine. Cca 50 mins for full import (from start SQL
> injection to searching in staff OPAC).
>
> It's my second migration to EG (previous in EG 2.0 times).
>
> Thanks for advance
>
> Vaclav Jansa
>
>
>
> On 10/15/2013 12:00 PM, Sharp, Chris wrote:
> > Hi Linda,
> >
> > This appears to be the problem:
> >
> > ERROR: could not extend file "base/42287/42883": No space left on
> > device
> > HINT: Check free disk space.
> >
> > You can verify disk space with the bash command 'df -h', but it
> > looks like you need to increase your disk space before continuing.
> >
> > Note that this is a PostgreSQL-level issue and is not specific to
> > Evergreen.
> >
> > Hope that's helpful!
> >
> > Chris
> >
> > ----- Original Message -----
> >> From: "Linda Jansova" <skolkova at chello.cz>
> >> To: "Evergreen Discussion Group"
> >> <open-ils-general at list.georgialibraries.org>
> >> Sent: Tuesday, October 15, 2013 4:33:28 AM
> >> Subject: [OPEN-ILS-GENERAL] ***SPAM*** Importing bib and holdings
> >> data into Evergreen 2.4.2
> >>
> >>
> >> Dear all,
> >>
> >> We have attempted to import bibliographic and holdings data into
> >> Evergreen 2.4.2 using instructions from chapter 14 (
> >> http://docs.evergreen-ils.org/2.4/_migrating_from_a_legacy_system.html
> >> ) of the official Evergreen documentation.
> >>
> >> First, we were unsuccessful in using the create function (step 3):
> >>
> >> ***
> >> CREATE OR REPLACE FUNCTION staging_importer() RETURNS NULL AS $$
> >> DECLARE stage RECORD;
> >> BEGIN
> >> FOR stage IN SELECT * FROM staging_records_import ORDER BY id LOOP
> >> INSERT INTO biblio.record_entry (marc, last_xact_id) VALUES
> >> (stage.marc, 'IMPORT');
> >> UPDATE staging_records_import SET dest =
> >> currval('biblio.record_entry_id_seq');
> >> END LOOP;
> >> END;
> >> $$ LANGUAGE plpgsql;
> >>
> >>
> >> ***
> >>
> >> However, it worked okay when we substituted NULL for VOID as
> >> recommended at
> >> http://comments.gmane.org/gmane.education.libraries.open-ils.general/7820
> >> . Then it took well over 12 hours to import 9460 records. Yet, the
> >> import was unsuccessful:
> >>
> >> ***
> >> evergreen=# CREATE TABLE staging_records_import (id BIGSERIAL,
> >> dest
> >> BIGINT, marc TEXT);
> >>
> >> evergreen=# CREATE OR REPLACE FUNCTION staging_importer() RETURNS
> >> VOID AS $$
> >> DECLARE stage RECORD;
> >> BEGIN
> >> FOR stage IN SELECT * FROM staging_records_import ORDER BY id LOOP
> >> INSERT INTO biblio.record_entry (marc, last_xact_id) VALUES
> >> (stage.marc, 'IMPORT');
> >> UPDATE staging_records_import SET dest =
> >> currval('biblio.record_entry_id_seq');
> >> END LOOP;
> >> END;
> >> $$ LANGUAGE plpgsql;
> >> CREATE FUNCTION
> >>
> >>
> >> evergreen=# COPY staging_records_import (marc) FROM
> >> '/home/opensrf/KKB/IMPORT.xml';
> >> COPY 9460
> >>
> >>
> >> evergreen=# SELECT staging_importer();
> >> ERROR: could not extend file "base/42287/42883": No space left on
> >> device
> >> HINT: Check free disk space.
> >> CONTEXT: SQL statement "INSERT INTO metabib.real_full_rec (record,
> >> tag, ind1, ind2, subfield, value)
> >> SELECT record, tag, ind1, ind2, subfield, value FROM
> >> biblio.flatten_marc( bib_id )"
> >> PL/pgSQL function "reingest_metabib_full_rec" line 7 at SQL
> >> statement
> >> SQL statement "SELECT metabib.reingest_metabib_full_rec(NEW.id)"
> >> PL/pgSQL function "indexing_ingest_or_delete" line 46 at PERFORM
> >> SQL statement "INSERT INTO biblio.record_entry (marc,
> >> last_xact_id)
> >> VALUES (stage.marc, 'IMPORT')"
> >> PL/pgSQL function "staging_importer" line 5 at SQL statement
> >>
> >> ***
> >>
> >> The statistics of the virtual machine load have showed that the in
> >> the first hour disk I/O write operation (at about 30 MBps)
> >> prevailed, while in the rest of the time disk I/O read operations
> >> (at about 70 MBps) were prevalent.
> >>
> >> When we tried to use the import method recommended for previous
> >> versions (especially version 2.1:
> >> http://docs.evergreen-ils.org/2.1/html/migrating_records_using_migration_tools.html
> >> ), the import of bibliographic data was successful (and it took
> >> only
> >> about an hour overall). However, when we subsequently wanted to
> >> import holdings data, the system was not able to fill the column
> >> entitled egid and therefore we could not import holdings data.
> >>
> >> Any ideas how to sort this out? Or which import methods to use for
> >> Evergreen 2.4.2?
> >>
> >> Thanks in advance for any hints!
> >>
> >> Linda and Vaclav Jansovi :-)
> >>
> >>
> >>
>
>
--
Chris Sharp
PINES System Administrator
Georgia Public Library Service
1800 Century Place, Suite 150
Atlanta, Georgia 30345
(404) 235-7147
csharp at georgialibraries.org
http://pines.georgialibraries.org/
More information about the Open-ils-general
mailing list