[OPEN-ILS-DEV] Book covers and encoding issues

[Linda Jansova]

Hi all,

Back in March 2016 we were awarded a grant to create a Perl module for 
Czech book covers (and related added content like tables of contents, 
summaries or book ratings) from obalkyknih.cz.

We have already created a working version of the module and implemented 
it on our test server (currently running Evergreen 2.10.2) at 
http://eg-test.osvobozena-knihovna.cz/.

A sample catalog record with added content from obalkyknih.cz can be 
found here:

http://www.jabok.cuni.cz/eg/opac/record/13907?locg=102

However, we are struggling to *get the encoding right*. It is a case 
with*all text-based added content* such as summaries or tables of 
contents in text format. Our programmer has made sure that the content 
received from obalkyknih.cz (the provider) is okay.

We have also *tested Open Library added content in Evergreen* which is a 
default added content provider in Evergreen (we haven't tested the other 
modules like Content Cafe as these are subscription-based).

To make sure we have a book that is both in our system and in Open 
Library, we have added an appropriate table of contents to this book:

https://openlibrary.org/works/OL577950W/U%CC%81stavni%CC%81_pe%CC%81c%CC%8Ce

When we check how it looks like in our production system (Evergreen 
2.8.3), we *get corrupted letters* as you can see at:

http://www.jabok.cuni.cz/eg/opac/record/13907?locg=102 (please click on 
Additional Content in the lower part of the page)

We have also *tested a 2.10.5 instance with Open Library data *and the 
problem has remained the same**(due to the lack of free public IP 
addresses this Evergreen I am unable to send you a working link).

Therefore we assume *the problem might not be in our module (as Open 
Library module shares the same issue) but maybe in AddedContent.pm*? Or 
somewhere else?

I am not a programmer myself but I have had a look at Evergreen Git and 
have come across the following possible hints (I may be thoroughly 
mistaken, though):

  * AddedContent.pm
    (http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/perlmods/lib/OpenILS/WWW/AddedContent.pm;h=254d790cf630242e6a07bc57286e56ec1a7afb34;hb=HEAD)
    mentions Base64 encoding (see line 340).
  * Table of contents is mentioned in the files at
    http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/xsl/MARC21slim2MODS32.xsl
    and at
    http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/xsl/MARC21slim2MODS33.xsl.
    (We already experienced a problem with these templates but at least
    for Zotero it has been - thanks to Dan Scott - resolved:
    https://bugs.launchpad.net/evergreen/+bug/1442276.)
  * Feed.pm module
    (http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/perlmods/lib/OpenILS/WWW/SuperCat/Feed.pm,
    line 796) and some .sql files seem to use the output_string which is
    probably deprecated as Dan pointed out in connection with Zotero
    encoding issues. The list of files which mention output_string can
    be found here:
    http://git.evergreen-ils.org/?p=Evergreen.git&a=search&h=HEAD&st=grep&s=output_string.

(BTW - of course once our module is polished and ready to share - which 
will happen rather soon -, we shall work with you so that it is included 
in Evergreen.)

Thank you in advance for your help in our programming endeavor!

Linda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20160804/9b75a7a4/attachment.html>