[OPEN-ILS-DEV] z39.50 again

[Mike Rylander]

On Thu, Sep 9, 2010 at 12:55 PM, Grant Johnson <fgjohnson at upei.ca> wrote:
> Yes
>  - forcing the LDR/09 to be "a" resolves the current issue.
> However - That isn't really a solution going forward.
> Records will keep coming in that require this adjustment which can't be
> dome at the client level.
>
> The updated SuperCat.pm didn't seem to resolve this issue for me...
> I'll verify that I have the correct one
>  - Is its purpose to "add an a" to position LDR  9 for all records as
> they are returned by a search?
>

It won't fix your existing data to do so, but if you update
MARC::File::XML to version 0.92 or better then new records imported
from non-UTF8 sources going forward will get the correct encoding
value in the leader:

0.91 Tue Jul 28 2009
      - Nixing the stray space (always been there) in the schemaLocation
        attribute for single-record output
      - Properly apply the leader modification for USMARC to signal UTF-8
        encoding.
      - Escape '<', '>', and '&' used as indicator values and subfield labels
        during XML output (Bill Erickson)
      - CPAN RT#34082: clarify names of header output switches

Note the second item.

--miker

>
>
>
> F. Grant Johnson
>  Systems Coordinator
>  309 Robertson Library
>  University of Prince Edward Island
>  902-566-0630(w),  902-393-4920(c)
>
>
>
>
>
>>>> On 9/7/2010 at 06:34 PM, in message
> <AANLkTi=kPcYeGDQWRViSE2u1i8hk1wWjHM0woo7QG1NW at mail.gmail.com>, Joe
> Atzberger
> <jatzberger at esilibrary.com> wrote:
>> Yup, classic problem where the record uses UTF8, but the leader
> doesn't know
>> it.  Fix the data (LDR/09), and the problem is solved.
>>
>> --Joe
>>
>> On Tue, Sep 7, 2010 at 5:28 PM, Dan Scott <dan at coffeecode.net>
> wrote:
>>
>>> On Tue, 2010-09-07 at 16:55 -0300, Grant Johnson wrote:
>>> > Thanks Dan,
>>> >
>>> > Same versions as indicated.
>>> >
>>> >  yaz-config --version
>>> > 3.0.47
>>> >
>>> > perl -MNet::Z3950::Simple2ZOOM -e 'print
>>> > > $Net::Z3950::Simple2ZOOM::VERSION . "\n"'
>>> > 1.04
>>> >
>>> >
>>> > I added  inputcharset="utf-8"  like so... to no avail... should
> this be
>>> > marc-8?
>>>
>>> Nope, everything coming out of Evergreen is UTF-8.
>>>
>>> If I've counted correctly, the record in question is
>>>
>>>
>>
> http://islandpines.roblib.upei.ca/opac/en-CA/skin/roblib/xml/rdetail.xml?r=58
>> 0632&ol=4&t=medicine&tp=keyword&l=4&d=2&o=60&hc=4019&rt=keyword
>>>
>>> To look at that a little closer:
>>>
>>>
>>
> http://islandpines.roblib.upei.ca/opac/extras/supercat/format/marcxml/record/
>> 580632
>>>
>>> And even closer, let's look at the leader:
>>>
>>> <leader>01607cam  2200373 a 4500</leader>
>>>
>>> Is LDR/09 an 'a', indicating UTF8 encoding? No sir! Does that
> confuse
>>> current versions of MARC::Record / MARC::File::XML no end, in that
> it
>>> sees that the record claims to be MARC-8 encoded (due to the space
>>> instead of the 'a' in LDR/09) and dutifully tries to convert the
> record
>>> from MARC8 to UTF8 (even though it's already in UTF8)?
>>>
>>> Down in the 505 $a, we have some \x2019 Unicode characters.
>>>
>>> Could that result in immense pain? Indeed! You can see where we ran
> into
>>> some of that pain in the past in the diffs at
>>> http://svn.open-ils.org/trac/ILS/changeset?new=16250%40branches%
>>> 2Frel_1_6_0%2FOpen-ILS%2Fsrc%2Fperlmods%2FOpenILS%2FWWW%
>>> 2FSuperCat.pm&old=13582%40branches%2Frel_1_6_0%2FOpen-ILS%2Fsrc%
>>> 2Fperlmods%2FOpenILS%2FWWW%2FSuperCat.pm - specifically in this
> instance
>>> forcing LDR/09 to 'a' before passing it through the potentially
>>> character-corrupting as_xml_record().
>>>
>>> Make sure you have an up-to-date copy of this
>>> (
>>>
>>
> http://svn.open-ils.org/trac/ILS/browser/branches/rel_1_6_0/Open-ILS/src/perl
>> mods/OpenILS/WWW/SuperCat.pm)
>>> in your /openils/lib/perl5/OpenILS/WWW/SuperCat.pm directory,
> restart
>>> apache, and give it a go again. I've got my fingers crossed that
> you'll have
>>> a better result.
>>>
>
>



-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com