[OPEN-ILS-GENERAL] [OPEN-ILS-DEV] Cannot save authority record with a 155 field

Sun Nov 11 13:54:01 EST 2018

A little update if someone else needs to perform the upgrade with the 
changed XSLT - the lines also have to be swapped in the following file:

http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/sql/Pg/version-upgrade/2.12.6-3.0.0-upgrade-db.sql;h=7a865a89db22c333e8da9e3e7750bc9d1fc8bb95;hb=HEAD

Linda

On 11/8/18 6:20 AM, Linda Jansova wrote:
> Hi,
>
> a little update - Nate Trail from the LC has promised to fix it:
>
> https://listserv.loc.gov/cgi-bin/wa?A2=MODS;692400d8.1811
>
> :-)
>
> Linda
>
> On 11/7/18 11:11 AM, Linda Jansova wrote:
>> Dear all,
>>
>> We have updated the XSLT (with swapped lines 1084 and 1085 as Josh 
>> suggested) using the following command: 
>> http://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/sql/Pg/955.data.MADS21-xsl.sql. 
>> Then we performed all steps that the upgrade from 2.12.6 to 3.1.4 
>> requires. Everything (including the reingest) has been completed 
>> successfully :-).
>>
>> On Monday November 5th, I sent a message to MODS (MADS) mailing list 
>> regarding the troublesome lines in the XSLT file:
>>
>> https://listserv.loc.gov/cgi-bin/wa?A2=MODS;fd3eba0f.1811
>>
>> There is no response yet but I do hope that someone from the LC will 
>> eventually respond.
>>
>> Linda
>>
>> On 11/1/18 7:29 AM, Linda Jansova wrote:
>>> Thank you, Josh!
>>>
>>> We shall try and go through the reingest of all records and if no 
>>> other XSLT-related issues appear, I can get in touch with MADS folks 
>>> through their listserv and report the error.
>>>
>>> Linda
>>>
>>> On 10/31/18 4:59 PM, Josh Stompro wrote:
>>>> Hello Mike, I've created a bug for this issue to get it in the 
>>>> queue.  LP#1800871
>>>>
>>>> Josh Stompro - LARL IT Director
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Open-ils-general 
>>>> <open-ils-general-bounces at list.georgialibraries.org> On Behalf Of 
>>>> Mike Rylander
>>>> Sent: Wednesday, October 31, 2018 10:09 AM
>>>> To: Evergreen Development Discussion List 
>>>> <open-ils-dev at list.georgialibraries.org>
>>>> Cc: Evergreen Discussion Group 
>>>> <open-ils-general at list.georgialibraries.org>
>>>> Subject: Re: [OPEN-ILS-GENERAL] [OPEN-ILS-DEV] Cannot save 
>>>> authority record with a 155 field
>>>>
>>>> Hi Linda and Josh,
>>>>
>>>> Great work figuring that out! We should definitely report this 
>>>> upstream to the MODS/MADS folks, but we maintain a local copy of 
>>>> the XSLT (altered to work without external file or network access 
>>>> for xml
>>>> includes) and can fix it locally.  I don't have time right now to 
>>>> jump on it, but both the file on the filesystem and the version in 
>>>> the config.xml_transform table in the db should be corrected.
>>>>
>>>> Thanks!
>>>> -- 
>>>> Mike Rylander
>>>>   | Executive Director
>>>>   | Equinox Open Library Initiative
>>>>   | phone:  1-877-OPEN-ILS (673-6457)
>>>>   | email:  miker at equinoxinitiative.org
>>>>   | web:  http://equinoxinitiative.org
>>>>
>>>> On Wed, Oct 31, 2018 at 10:58 AM Josh Stompro 
>>>> <stomproj at exchange.larl.org> wrote:
>>>>> Linda, I wonder if this is a bug in the MARCslim2MADS.xslt?  The 
>>>>> error
>>>>> message is
>>>>>
>>>>>
>>>>>
>>>>> runtime error: file ./MARC21slim2MADS.xsl line 1404 element attribute
>>>>>
>>>>> xsl:attribute: Cannot add attributes to an element if children 
>>>>> have been already added to the element.
>>>>>
>>>>>
>>>>>
>>>>> I agree that it looks like the problem is when the xslt is 
>>>>> processing the 755 and trying to set the authority source.
>>>>>
>>>>>
>>>>>
>>>>> I think the bug may be that when processing the 755 tag on line 
>>>>> 1081, the genre template is called before the setAuthority 
>>>>> template.  The Genre template adds child elements, then the 
>>>>> setAuthority tries to set attributes, which is where the error 
>>>>> pops up.
>>>>>
>>>>>
>>>>>
>>>>> If I swap lines 1084 and 1085 then the error goes away and both 
>>>>> the genre(155) and related genre(755) show up in the transformed xml.
>>>>>
>>>>>
>>>>>
>>>>> I think the way to report this to the MADS project is via the MODS
>>>>> listserv, as listed on http://www.loc.gov/standards/mads/
>>>>>
>>>>>
>>>>>
>>>>> I don’t have enough experience with these technologies to be all 
>>>>> that confident that this is the issue though.  I would be happy to 
>>>>> report this to the MODS listserv if it seems to make sense to 
>>>>> someone that is more familiar with mods/mads/xml/authorities.
>>>>>
>>>>>
>>>>>
>>>>> Josh Stompro - LARL IT Director
>>>>>
>>>>>
>>>>>
>>>>> From: Linda Jansova <skolkova at chello.cz>
>>>>> Sent: Wednesday, October 31, 2018 7:16 AM
>>>>> To: Evergreen Discussion Group
>>>>> <open-ils-general at list.georgialibraries.org>; Josh Stompro
>>>>> <stomproj at exchange.larl.org>; Evergreen Development Discussion List
>>>>> <open-ils-dev at list.georgialibraries.org>
>>>>> Subject: Re: [OPEN-ILS-GENERAL] Cannot save authority record with a
>>>>> 155 field
>>>>>
>>>>>
>>>>>
>>>>> Dear Josh,
>>>>>
>>>>> Thank you for letting me know about the right XSL file!
>>>>>
>>>>> After some more investigations I have come to a conclusion that it 
>>>>> is not actually a 155 field which causes the problem but a 755 
>>>>> field. If it has any value in second indicator, xsltproc fails to 
>>>>> process it. When the indicator does not have any value (or, to be 
>>>>> more precise, there is just a space), it is okay.
>>>>>
>>>>> So far, it seems that we will have to get rid either of the values 
>>>>> of indicators in 755s, or of the following part of the XSL file:
>>>>>
>>>>> <xsl:when
>>>>>                  test="(700 <= 
>>>>> ancestor-or-self::marc:datafield/@tag and 
>>>>> ancestor-or-self::marc:datafield/@tag <= 755 ) and @ind2='7'">
>>>>>                  <xsl:attribute name="authority">
>>>>>                      <xsl:value-of 
>>>>> select="marc:subfield[@code='2']"/>
>>>>>                  </xsl:attribute>
>>>>>              </xsl:when>
>>>>>
>>>>> Hopefully it will work for us and let us proceed in the upgrade :-)!
>>>>>
>>>>> Linda
>>>>>
>>>>> On 10/30/18 9:38 PM, Josh Stompro wrote:
>>>>>
>>>>> Hello Linda, I think the Authority ingest uses the 
>>>>> MARC21slim2MADS.xsl transform file to convert the authority data 
>>>>> into MADS format.  Could you try manually processing your problem 
>>>>> authority record using the MADS file instead of the MODS and see 
>>>>> what you get.
>>>>>
>>>>>
>>>>>
>>>>> The MADS xsl does look like it references tag 155.
>>>>>
>>>>>
>>>>>
>>>>> Josh Stompro - LARL IT Director
>>>>>
>>>>>
>>>>>
>>>>> From: Open-ils-general
>>>>> <open-ils-general-bounces at list.georgialibraries.org> On Behalf Of
>>>>> Linda Jansova
>>>>> Sent: Friday, October 26, 2018 5:29 AM
>>>>> To: Evergreen Discussion Group
>>>>> <open-ils-general at list.georgialibraries.org>; Evergreen Development
>>>>> Discussion List <open-ils-dev at list.georgialibraries.org>
>>>>> Subject: [OPEN-ILS-GENERAL] Cannot save authority record with a 155
>>>>> field
>>>>>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> back in August we started investigating why we couldn't proceed 
>>>>> with upgrade from 2.12.6 to 3.1.4 (for more details please see 
>>>>> http://libmail.georgialibraries.org/pipermail/open-ils-general/2018-August/015298.html).
>>>>>
>>>>> After removing obviously invalid MARCXML records (which 
>>>>> surprisingly made their way to our 2.12 installation) we still 
>>>>> have some records which cannot be reingested (or saved).
>>>>>
>>>>> Attached is a sample record americke_romany.xml which is one of 
>>>>> those troublesome ones. It is a genre/form term record with the 
>>>>> main heading in the field 155.
>>>>>
>>>>> We have tried the SQL upgrade from 2.12.6 to 3.0.0 without 
>>>>> authority records reingest (the particular lines were commented 
>>>>> out) and, once we were at 3.1.4, used the web client to save this 
>>>>> particular record (without actually making any changes in it). 
>>>>> However, it appeared that it could not be saved:
>>>>>
>>>>> ---
>>>>>
>>>>> open-ils.pcrud 2018-10-26 09:33:37 [ERR
>>>>> :49144:oils_sql.c:6570:15405352984867814] open-ils.pcrud ERROR
>>>>> updating authority::record_entry object with id = 356: 56966976
>>>>> 56966976: ERROR: runtime error: file unknown-55cee6a934f0 element
>>>>> attribute
>>>>>
>>>>> xsl:attribute: Cannot add attributes to an element if children 
>>>>> have been already added to the element.
>>>>>
>>>>> at line 31.
>>>>>
>>>>> CONTEXT: PL/Perl function "oils_xslt_process"
>>>>>
>>>>> ---
>>>>>
>>>>> Using this error message, we began to suspect a XSLT 
>>>>> transformation being the culprit. We have taken XSL files from 
>>>>> Evergreen (those from 
>>>>> http://git.evergreen-ils.org/?p=Evergreen.git;a=tree;f=Open-ILS/xsl;h=68fd13ffb2ad01ef9ceacf9f18695f25d284df05;hb=HEAD). 
>>>>> When they were used (xsltproc MARC21slim2MODS33.xsl 
>>>>> americke_romany.xml > output.xml), the contents of the 155 field 
>>>>> (which is a heading and therefore one of the most important parts 
>>>>> of the record) was never included in the output (please see the 
>>>>> attached output.xml file).
>>>>>
>>>>> Then we used a web client again to change the 155 tag to the 100 
>>>>> tag (and deleted another possible troublesome tag 755). After 
>>>>> making these changes, the record could be saved.
>>>>>
>>>>> So the question is:
>>>>>
>>>>> Where should we add the 155 field (probably in which of the XSLT 
>>>>> files) to make sure those records can be saved (or of course 
>>>>> reingested)?
>>>>>
>>>>> Thank you in advance for any hints!
>>>>>
>>>>> Linda
>>>
>>
>