[OPEN-ILS-DEV] alternative to quick_metarecord_map.sql

[Dan Scott]

2008/6/20 Bill Ott <bott at grpl.org>:
> Last question of the week:
>
> If importing a small batch of records into a database with a few hundred
> thousand existing records, is there a good alternative to running
> quick_metarecord_map.sql?    ...it can get a bit time consuming.

Untested, but theoretically running the two INSERT statements within a
transaction limiting to the biblio.record_entry IDs (WHERE id >
somenumber) rather than rebuilding the whole thing from scratch would
avoid the full overhead:

BEGIN;

INSERT INTO metabib.metarecord (fingerprint,master_record)
	SELECT	fingerprint,id
	  FROM	(SELECT	DISTINCT ON (fingerprint)
	  		fingerprint, id, quality
		  FROM	biblio.record_entry
		  WHERE id > somenumber
		  ORDER BY fingerprint, quality desc) AS x
	  WHERE	fingerprint IS NOT NULL;

INSERT INTO metabib.metarecord_source_map (metarecord,source)
	SELECT	m.id, b.id
	  FROM	biblio.record_entry b
	  	JOIN metabib.metarecord m ON (m.fingerprint = b.fingerprint)
	  WHERE b.id > somenumber;

COMMIT;

Alternately, Jason just pointed me to the (slightly different) version
of this same idea at the very bottom of
http://open-ils.org/dokuwiki/doku.php?id=scratchpad:random_magic_spells
- it's more likely that that's tested, and the AND NOT EXISTS clause,
upon reflection, is an intelligent addition :)

One more update: Mike states that the "magic spell" has been used many
times. So go with that.



-- 
Dan Scott
Laurentian University