[OPEN-ILS-GENERAL] Data migration – Best practices

[Justin Hopkins]

Good morning Donald,

On Fri Mar  8 07:52:19 2013, Donald Butterworth wrote:
> Colleagues,
>
>
> We are approaching the point of migrating records from our Symphony
> system into Evergreen, and we are unsure what the best method is to
> accomplish this.
>

We have migrated several different ILS systems to Evergreen. In fact, 
our next library is running Symphony. Actually, I've not had a chance 
to look at their data in detail and am a bit perplexed about how to 
neatly retrieve a csv of patron information and/or circulations...

We use the same general approach to migrations no matter what the 
system - we follow the community documentation (the 2.1 version 
specifically) but have adapted/added to it as we've gone along. We also 
got a great jump start by visiting the folks at Evergreen Indiana (who 
have 100 or so libraries online) and checking out how they do it.

> Based on insights from the listserv and some serious cogitation, we
> have created hierarchies, parameters, and values that we hope will
> take advantage of Evergreen’s architecture. All along my expectation
> has been that we would export records from Symphony, take the data
> found in the 999 tag and massage it until it matches the values of our
> new Evergreen structures, and then do an import. However, recently it
> has been suggested that a better method would be to take the
> structures and values we currently have in Symphony and duplicate them
> (as much as possible) in Evergreen. Then once, the data is in
> Evergreen, change the values into what we really want them to be.

If you aren't happy with your setup in Symphony you probably won't like 
it in Evergreen. Massaging the data _is_ work, but it could prove to be 
easier than trying to shoehorn Symphony's setup into Evergreen. We 
massage.

> I have to say that method number two make me very nervous because I
> remember that making global changes in our former “Horizon” system was
> virtually impossible. Here are some of the questions we are confronting:
>
> ·* What is the best way to simply re-label an Org Unit? For example
> changing the value ATSFLA to ATS-DCL?
>
> ·* What about circulation modifiers? What is the best approach to
> changing the values DVD to dvd, FAC-MED to dvd, and ARC-MED to dvd?
>
> ·* In Symphony we currently have two “Libraries” one for the main
> campus and one for a branch. But in Evergreen we want to take
> advantage of the “sub-library” option for our Archives and two other
> satellite collections. The Symphony “match points” that can be used to
> identify these sub-libraries are called “Home Location” and “Current
> Location”. What would be the best way to approach this problem, short
> of a light gun and a laptop?

During migrations we handle all of these sorts of things with "mapping 
documents". During our initial implementation discussions with the 
library we create a set of spreadsheets (or we go back later with 
sheets based on what we find in the data). These spreadsheets are used 
to map individual legacy "columns" or combinations of columns to their 
Evergreen equivalents. Then it's just spreadsheet -> tsv -> postgres 
:-) It's easier than you'd think to collect this information and once 
you've got it - and your item information - into staging tables on the 
database you can run queries to update all of your items very quickly.

> ·       * Is it true that all bibliographic records must have attached
> item/copy records in order for the bib record to appear in the public
> PAC? We have several thousand “analytic” records that have no attached
> item/copy records.

If the items have an 856 field, then you can use the "located URI's" 
feature. By adding a subfield 9 (or multiples) to the 856's containing 
the shortname of an org unit it will cause it to appear in OPAC 
searches despite the lack of physical copies.

The other method to show itemless records in the OPAC is by making them 
"transcendent". I'm a little fuzzy on this one unfortunately because we 
don't use it. This works for records without an 856, but will show in 
the entire OPAC rather than only in selected org units. You need to 
create a new "bib source" and turn on transcendency for that source. 
Then you'd set the source of your "analytic" records to use this 
source. At the time we looked at this feature you had to create the new 
source by adding a row to the config.bib_source table of the database 
but there may be a UI for it now in the staff client.

Regards,
Justin Hopkins
MOBIUS
573-808-2309