[OPEN-ILS-DEV] IDL is not backward compatible?

[Ken Cox]

Dan Wells: Good suggestion re: opensrf.open-ils.system.ils_version.  I
will think more about it.  For now, disabling caching for the
fm_IDL.xml request will have the same effect as querying the version
on every login. It would help if auth tokens got invalidated during
server upgrades; does anyone know whether that's true?

Dan Scott: Good point about cache-busting params; now that I know to
do it I can avoid caching and save 72KB in the cache for something
useful.  I didn't know about /IDL2js, thanks.  The URL I use is:

http://bark.cwmars.org/reports/fm_IDL.xml?class=ac&class=acn&class=acp&class=ahr&class=ahtc&class=aou&class=aout&class=au&class=aua&class=auact&class=aum&class=aus&class=bmp&class=cbreb&class=cbrebi&class=cbrebin&class=cbrebn&class=ccs&class=circ&class=csc&class=cuat&class=ex&class=mbt&class=mbts&class=mous&class=mra&class=mraf&class=mus&class=mvr&class=perm_ex

I'm not sure whose idea adding all the 'class' params was (probably
Dan Wells) but that response is 72KB whereas the full IDL is 820KB,
and since I don't need the reset, it saves a lot of time parsing.

Regards,
Ken

On Mon, Feb 18, 2019 at 10:14 PM Dan Scott <dan at coffeecode.net> wrote:
>
> Hi Ken:
>
> In addition to the headers you're getting back, it might be helpful to include the exact GET request you're making. If you're referring to /IDL2js, then that's ~400kb of data that's being shipped--a not inconsiderable amount to send over the wire.
>
> Back in 2017, Evergreen's web environment adopted the generally accepted performance approach of setting the cache time as long as possible and relying on cache-busting GET params to only cause browsers to request new versions of files when things actually change. When things change (for example, when a server is upgraded and autogen.sh is run), the GET param changes from ?abc123 to ?def456 (or some other such hash), breaking the cache and forcing a fresh retrieval of the file in question.
>
> Prior to commit 1cb0d8c63c44, the default caching time was 18 hours. So crashes due to IDL mismatches could still occur if the cache-busting GET param wasn't updated; the window was just shorter.
>
> It sounds like in the the iOS app, if you want to force a refetch of the cached file, perhaps you could generate a similar cache-busting token and just append it to your GET URL?
>
> Best,
> Dan
>
> Best,
> Dan
>
> Dan
>
> On Mon, Feb 18, 2019 at 6:12 PM Ken Cox <kenstir at gmail.com> wrote:
>>
>> I think I know what is going on, but I would like somebody to confirm
>> my understanding.
>>
>> After a server upgrade from 3.0.1 to 3.2.2, I saw a significant spike
>> the iOS app crashes.  It looks like a cached IDL specifies fewer
>> fields than is returned by the /osrf-gateway-v1 endpoint.
>>
>> I fixed the app so that does not cause a crash, reasoning that since
>> the IDL fields must be newly added fields, and because the app didn't
>> require them, I could safely ignore them.  On deeper inspection, it
>> looks as though new fields have been added to the middle of IDL
>> classes.  For instance, the class 'au' had the following new fields
>> added, all in the middle:
>> * Privacy Waiver Entries
>> * Preferred Prefix
>> * Preferred First Name
>> ...
>>
>> Short term, I think a partial fix is to disable caching of the IDL,
>> even though the server indicates it can be cached:
>>
>>     HTTP/1.1 200 OK
>>     Date: Mon, 18 Feb 2019 09:27:03 GMT
>>     Cache-Control: max-age=31536000
>>     Expires: Tue, 18 Feb 2020 09:27:03 GMT
>>
>> Long term, I think a better approach would be to add new fields only
>> at the end.  This is a common technique for maintaining binary
>> compatibility in compiled code.  If the app is suspended during a
>> server upgrade, and if fields are added in the middle, then the IDL is
>> out of sync, right?  Or maybe, all sessions are lost during the down
>> time and the next request with an auth token will fail, and the IDL
>> should be loaded on every login?  (Sorry I have been trying to test
>> this hypothesis but I am having trouble upgrading my own EG server).
>>
>> Thanks,
>> Ken



-- 
-Ken