[OPEN-ILS-GENERAL] ***SPAM*** Importing bib and holdings data into Evergreen 2.4.2

[Sharp, Chris]

Hi Linda,

This appears to be the problem:

ERROR:  could not extend file "base/42287/42883": No space left on device
HINT:  Check free disk space.

You can verify disk space with the bash command 'df -h', but it looks like you need to increase your disk space before continuing.

Note that this is a PostgreSQL-level issue and is not specific to Evergreen.

Hope that's helpful!

Chris

----- Original Message -----
> From: "Linda Jansova" <skolkova at chello.cz>
> To: "Evergreen Discussion Group" <open-ils-general at list.georgialibraries.org>
> Sent: Tuesday, October 15, 2013 4:33:28 AM
> Subject: [OPEN-ILS-GENERAL] ***SPAM*** Importing bib and holdings data into	Evergreen 2.4.2
> 
> 
> Dear all,
> 
> We have attempted to import bibliographic and holdings data into
> Evergreen 2.4.2 using instructions from chapter 14 (
> http://docs.evergreen-ils.org/2.4/_migrating_from_a_legacy_system.html
> ) of the official Evergreen documentation.
> 
> First, we were unsuccessful in using the create function (step 3):
> 
> ***
> CREATE OR REPLACE FUNCTION staging_importer() RETURNS NULL AS $$
> DECLARE stage RECORD;
> BEGIN
> FOR stage IN SELECT * FROM staging_records_import ORDER BY id LOOP
>       INSERT INTO biblio.record_entry (marc, last_xact_id) VALUES
>       (stage.marc, 'IMPORT');
>       UPDATE staging_records_import SET dest =
>       currval('biblio.record_entry_id_seq');
>    END LOOP;
>   END;
>   $$ LANGUAGE plpgsql;
> 
> 
> ***
> 
> However, it worked okay when we substituted NULL for VOID as
> recommended at
> http://comments.gmane.org/gmane.education.libraries.open-ils.general/7820
> . Then it took well over 12 hours to import 9460 records. Yet, the
> import was unsuccessful:
> 
> ***
> evergreen=# CREATE TABLE staging_records_import (id BIGSERIAL, dest
> BIGINT, marc TEXT);
> 
> evergreen=# CREATE OR REPLACE FUNCTION staging_importer() RETURNS
> VOID AS $$
> DECLARE stage RECORD;
> BEGIN
> FOR stage IN SELECT * FROM staging_records_import ORDER BY id LOOP
> INSERT INTO biblio.record_entry (marc, last_xact_id) VALUES
> (stage.marc, 'IMPORT');
> UPDATE staging_records_import SET dest =
> currval('biblio.record_entry_id_seq');
> END LOOP;
> END;
> $$ LANGUAGE plpgsql;
> CREATE FUNCTION
> 
> 
> evergreen=# COPY staging_records_import (marc) FROM
> '/home/opensrf/KKB/IMPORT.xml';
> COPY 9460
> 
> 
> evergreen=# SELECT staging_importer();
> ERROR: could not extend file "base/42287/42883": No space left on
> device
> HINT: Check free disk space.
> CONTEXT: SQL statement "INSERT INTO metabib.real_full_rec (record,
> tag, ind1, ind2, subfield, value)
> SELECT record, tag, ind1, ind2, subfield, value FROM
> biblio.flatten_marc( bib_id )"
> PL/pgSQL function "reingest_metabib_full_rec" line 7 at SQL statement
> SQL statement "SELECT metabib.reingest_metabib_full_rec(NEW.id)"
> PL/pgSQL function "indexing_ingest_or_delete" line 46 at PERFORM
> SQL statement "INSERT INTO biblio.record_entry (marc, last_xact_id)
> VALUES (stage.marc, 'IMPORT')"
> PL/pgSQL function "staging_importer" line 5 at SQL statement
> 
> ***
> 
> The statistics of the virtual machine load have showed that the in
> the first hour disk I/O write operation (at about 30 MBps)
> prevailed, while in the rest of the time disk I/O read operations
> (at about 70 MBps) were prevalent.
> 
> When we tried to use the import method recommended for previous
> versions (especially version 2.1:
> http://docs.evergreen-ils.org/2.1/html/migrating_records_using_migration_tools.html
> ), the import of bibliographic data was successful (and it took only
> about an hour overall). However, when we subsequently wanted to
> import holdings data, the system was not able to fill the column
> entitled egid and therefore we could not import holdings data.
> 
> Any ideas how to sort this out? Or which import methods to use for
> Evergreen 2.4.2?
> 
> Thanks in advance for any hints!
> 
> Linda and Vaclav Jansovi :-)
> 
> 
> 

-- 
Chris Sharp
PINES System Administrator
Georgia Public Library Service
1800 Century Place, Suite 150
Atlanta, Georgia 30345
(404) 235-7147
csharp at georgialibraries.org
http://pines.georgialibraries.org/