[OPEN-ILS-DEV] problem regarding bulk import

Thu Oct 30 09:16:31 EDT 2008

Dear sir,
Finally I am getting some error but follow the methods that you have suggested.

1) Last I knew, MARC::File::XML does not support namespace prefixes
("marc: ").  You can tell MarcEdit to leave them out by selecting
Tools->Preferences, select the MARCEngine tab, then uncheck "Use
Namespace" and apply.

2) If you are not trying to preserve record ids from a previous ILS,
you can forget about the 035 field and let marc2bre.pl generate valid
ids for your records.  I suggest grabbing the latest version from
http://svn.open-ils.org/trac/ILS/export/10967/trunk/Open-ILS/src/extras/import/marc2bre.pl
,

Thanks a lot,
And suggest me for further
with regards,
vijay kumar

((((((((((((((((((error))))))))))))))))))

root at test-evergreen:/home/test/Desktop/import_demo# sh import_bibs.sh
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
Use of uninitialized value in exists at marc2bre.pl line 152.
5001924.9498374421root at test-evergreen:/home/test/Desktop/import_demo#
sh import_bibs.sh

!!! Record 1 has missing or invalid id field 035, assinging new id.

!!! Record 2 has missing or invalid id field 035, assinging new id.

!!! Record 3 has missing or invalid id field 035, assinging new id.

!!! Record 4 has missing or invalid id field 035, assinging new id.

!!! Record 5 has missing or invalid id field 035, assinging new id.

!!! Record 6 has missing or invalid id field 035, assinging new id.

!!! Record 7 has missing or invalid id field 035, assinging new id.

!!! Record 8 has missing or invalid id field 035, assinging new id.

!!! Record 9 has missing or invalid id field 035, assinging new id.

!!! Record 10 has missing or invalid id field 035, assinging new id.

!!! Record 11 has missing or invalid id field 035, assinging new id.

!!! Record 12 has missing or invalid id field 035, assinging new id.

!!! Record 13 has missing or invalid id field 035, assinging new id.

!!! Record 14 has missing or invalid id field 035, assinging new id.

!!! Record 15 has missing or invalid id field 035, assinging new id.

!!! Record 16 has missing or invalid id field 035, assinging new id.

!!! Record 17 has missing or invalid id field 035, assinging new id.

!!! Record 18 has missing or invalid id field 035, assinging new id.

!!! Record 19 has missing or invalid id field 035, assinging new id.

!!! Record 20 has missing or invalid id field 035, assinging new id.

!!! Record 21 has missing or invalid id field 035, assinging new id.

!!! Record 22 has missing or invalid id field 035, assinging new id.

!!! Record 23 has missing or invalid id field 035, assinging new id.

!!! Record 24 has missing or invalid id field 035, assinging new id.

!!! Record 25 has missing or invalid id field 035, assinging new id.
5001910.51911883807root at test-evergreen:/home/test/Desktop/import_demo#
vim pg_loader-output.sql
root at test-evergreen:/home/test/Desktop/import_demo# su evergreen
evergreen at test-evergreen:/home/test/Desktop/import_demo$ pwd
/home/test/Desktop/import_demo
evergreen at test-evergreen:/home/test/Desktop/import_demo$ psql -U
evergreen -f pg_loader-output.sql evergreen
SET
BEGIN
COPY 25
COPY 25
COPY 25
COPY 25
COPY 26
COPY 25
COPY 616
COPY 3
COMMIT
evergreen at test-evergreen:/home/test/Desktop/import_demo$ psql -U
evergreen -f quick_metarecord_map.sql evergreen
BEGIN
ALTER TABLE
TRUNCATE TABLE
TRUNCATE TABLE
INSERT 0 482
INSERT 0 601
ALTER TABLE
COMMIT
psql:quick_metarecord_map.sql:25: INFO:  vacuuming "metabib.metarecord"
psql:quick_metarecord_map.sql:25: INFO:  "metarecord": found 0
removable, 482 nonremovable row versions in 6 pages
DETAIL:  0 dead row versions cannot be removed yet.
Nonremovable row versions range from 48 to 208 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 5668 bytes.
0 pages are or will become empty, including 0 at the end of the table.
5 pages containing 5640 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index "metarecord_pkey" now
contains 482 row versions in 4 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index
"metabib_metarecord_master_record_idx" now contains 482 row versions
in 5 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index
"metabib_metarecord_fingerprint_idx" now contains 482 row versions in
6 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  "metarecord": moved 1 row
versions, truncated 6 to 6 pages
DETAIL:  CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index "metarecord_pkey" now
contains 482 row versions in 4 pages
DETAIL:  1 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index
"metabib_metarecord_master_record_idx" now contains 482 row versions
in 5 pages
DETAIL:  1 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index
"metabib_metarecord_fingerprint_idx" now contains 482 row versions in
6 pages
DETAIL:  1 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  vacuuming "pg_toast.pg_toast_17668"
psql:quick_metarecord_map.sql:25: INFO:  "pg_toast_17668": found 0
removable, 0 nonremovable row versions in 0 pages
DETAIL:  0 dead row versions cannot be removed yet.
Nonremovable row versions range from 0 to 0 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 0 bytes.
0 pages are or will become empty, including 0 at the end of the table.
0 pages containing 0 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  index "pg_toast_17668_index"
now contains 0 row versions in 1 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:25: INFO:  analyzing "metabib.metarecord"
psql:quick_metarecord_map.sql:25: INFO:  "metarecord": scanned 6 of 6
pages, containing 482 live rows and 0 dead rows; 482 rows in sample,
482 estimated total rows
VACUUM
psql:quick_metarecord_map.sql:26: INFO:  vacuuming
"metabib.metarecord_source_map"
psql:quick_metarecord_map.sql:26: INFO:  "metarecord_source_map":
found 0 removable, 601 nonremovable row versions in 5 pages
DETAIL:  0 dead row versions cannot be removed yet.
Nonremovable row versions range from 52 to 52 bytes long.
There were 0 unused item pointers.
Total free space (including removable row versions) is 7204 bytes.
0 pages are or will become empty, including 0 at the end of the table.
5 pages containing 7204 free bytes are potential move destinations.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:26: INFO:  index
"metarecord_source_map_pkey" now contains 601 row versions in 4 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:26: INFO:  index
"metabib_metarecord_source_map_metarecord_idx" now contains 601 row
versions in 5 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:26: INFO:  index
"metabib_metarecord_source_map_source_record_idx" now contains 601 row
versions in 4 pages
DETAIL:  0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:26: INFO:  "metarecord_source_map":
moved 0 row versions, truncated 5 to 5 pages
DETAIL:  CPU 0.00s/0.00u sec elapsed 0.00 sec.
psql:quick_metarecord_map.sql:26: INFO:  analyzing
"metabib.metarecord_source_map"
psql:quick_metarecord_map.sql:26: INFO:  "metarecord_source_map":
scanned 5 of 5 pages, containing 601 live rows and 0 dead rows; 601
rows in sample, 601 estimated total rows
VACUUM
evergreen at test-evergreen:/home/test/Desktop/import_demo$ exit
exit
root at test-evergreen:/home/test/Desktop/import_demo# perl parse_holdings.pl
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 93.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in substitution (s///) at parse_holdings.pl line 44.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
Use of uninitialized value in printf at parse_holdings.pl line 75.
root at test-evergreen:/home/test/Desktop/import_demo# su evergreen
evergreen at test-evergreen:/home/test/Desktop/import_demo$ psql -U
evergreen -f import_staging_table.sql evergreen
psql:import_staging_table.sql:9: ERROR:  relation "staging_items" already exists
evergreen at test-evergreen:/home/test/Desktop/import_demo$ psql -U
evergreen -f insert_holdings.sql evergreen
psql:insert_holdings.sql:1: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:2: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:3: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:4: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:5: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:6: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:7: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:8: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:9: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:10: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:11: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:12: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:13: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:14: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:15: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:16: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:17: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:18: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:19: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:20: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:21: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:22: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:23: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:24: ERROR:  invalid input syntax for integer: ""
psql:insert_holdings.sql:25: ERROR:  invalid input syntax for integer: ""
evergreen at test-evergreen:/home/test/Desktop/import_demo$ psql -U
evergreen -f generate_copies.sql evergreen
INSERT 0 27
psql:generate_copies.sql:12: ERROR:  duplicate key violates unique
constraint "asset_call_number_label_once_per_lib"
psql:generate_copies.sql:47: ERROR:  duplicate key violates unique
constraint "copy_barcode_key"
evergreen at test-evergreen:/home/test/Desktop/import_demo$