[OPEN-ILS-GENERAL] Loading Batches of Patrons

[Martha Driscoll]

Hi Carrie,
We told our colleges to provide the patron data in a comma separated 
file with quotes around each data element to avoid problems with commas 
in addresses.  We gave them the specific order we wanted the data in as 
well as the code to use for the permission profile and home_ou.  If a 
field is blank, for example street2, they need to provide a null field 
(""). If the data files are not perfect, it's easy enough to fix them 
with a perl script, but most colleges provide perfect files.

We load the data into a staging table and then update the staging table 
with the database id number if the patron is already on-file.  We match 
on ident_value which is where we store the university ID.  We then run a 
script to update the on-file patron records which updates name, address, 
dob, phone, email, and expiration date.  Once the updates are done, we 
remove those records from the staging table and load the new records 
which remain using scripts very similar to the ones in the Evergreen 
documentation.

A couple of issues we have run into:

Some files need to be converted to uft8 because of accents:
   iconv -f ISO-8859-2 -t UTF-8 stufent-file > student-file-utf8

Some students get hand-input into Evergreen before we load.  If the 
library did not include the university ID, then we end up with duplicate 
records or barcode collisions.

Once the scripts are setup, record loading works quite well.  Some 
libraries provide one file per semester while others provide files every 
few days until the flurry of registration is over.  I load about 40-50 
files of students/staff/faculty per semester for 9 institutions.

Martha Driscoll
Systems Manager
North of Boston Library Exchange
Danvers, Massachusetts
www.noblenet.org

On 10/2/2013 4:42 PM, Curie, Carrie L wrote:
> Hi Everyone,
>
> Is there anyone out there loading batches of patrons?  If so, which
> external systems are you extracting the patron data from?  Are you able
> to share what scripts you run on the data?
>
> Thanks!
>
> Carrie
>
> Carrie Curie
>
> Support and Training Specialist
>
> PALS
>
> A Program of the Minnesota State Colleges and Universities
>
> Minnesota State University, Mankato
>
> 3022 Memorial Library
>
> Mankato, MN  56001
>
> Phone 507.389.1976
>
> Carrie.curie at mnsu.edu <mailto:Carrie.curie at mnsu.edu>
>