[OPEN-ILS-DEV] Which OpenSRF gateway should I use?

[Bill Erickson]

On Thu, Sep 22, 2016 at 10:23 AM, Mike Rylander <mrylander at gmail.com> wrote:

> On Wed, Sep 21, 2016 at 3:36 PM, Bill Erickson <berickxx at gmail.com> wrote:
> >
> > On Wed, Sep 21, 2016 at 1:01 PM, Mike Rylander <mrylander at gmail.com>
> wrote:
> >>
> >> On Wed, Sep 21, 2016 at 11:51 AM, Bill Erickson <berickxx at gmail.com>
> >> wrote:
> >> >
> >> > On Wed, Sep 21, 2016 at 10:44 AM, Bill Erickson <berickxx at gmail.com>
> >> > wrote:
> >> >>
> >> >>
> >> >> /osrf-http-translator provides a lower level OpenSRF API and supports
> >> >> streaming responses, but its use is no longer encouraged, because the
> >> >> underlying mechanism (multipart/x-mixed-replace) is deprecated and
> only
> >> >> works with our old version of XULRunner.  This interface will likely
> be
> >> >> deprecated along with the XUL client.
> >> >>
> >> >
> >> > Couple of additional comments for those following along...
> >> >
> >> > 1. The multipart/x-mixed-replace part of the translator only affects
> its
> >> > ability to stream results.  Without this, it's less efficient / fast,
> >> > but
> >> > still works.  You just can't use it for API calls that return large
> >> > result
> >> > sets or for real-time use of individual results.  The OpenSRF
> >> > API/session
> >> > support works regardless.
> >> >
> >> > 2. Various HTML interfaces outside of the XUL client proper (primarily
> >> > ACQ
> >> > and admin UI's) use the translator.  All of these would have to be
> >> > ported to
> >> > websockets before we could consider deprecation.
> >> >
> >>
> >> Meaning, in the end, it'll be a while before the translator goes away,
> >> or is even deprecated except for streaming.
> >>
> >> In addition to the core users, it's simpler (admin- and code-wise) on
> >> both ends than WebSockets for little things and has the benefit of
> >> being able to supply transactional semantics even in a clustered
> >> environment, which I don't think our WebSockets implementation can do
> >> today (though, if it can, please do correct me!).
> >
> >
> > Clustered transactions should work much the same way with Websockets
> with a
> > few caveats.
> >
> > 1. Apache KeepAlive does not affect WS connections, so clients will not
> be
> > required to bounce between Apache servers mid-session after a (default) 1
> > second timeout.  There is a WS idle timeout setting, but it's intended to
> > drop clients after minutes of inactivity, not seconds.
> >
>
> Well, that's configurable on the server, right?  I can imagine cases
> where it would be beneficial to have a shorter inactivity timeout.
> Even with the much smaller RAM requirements, a large site could have
> 1500+ essentially quiescent connections.
>

Yes, that's locally configurable.  There's a handy list of the settings at
step 6 in the install docs:

http://evergreen-ils.org/documentation/install/OpenSRF/README_2_4_1.html#_optional_websockets_installation_instructions

To avoid chopping any sessions off at the knees,
OSRF_WEBSOCKET_IDLE_TIMEOUT should be at bare minimum higher than the
highest drone keepalive value from opensrf.xml.  (The idle timeout only
comes into play when there are no in-flight requests, similar to OpenSRF
proper).  Beyond that, it is (as you suggest) a question of balancing
server resources with the desire to reduce the number of TCP
connect/disconnect's required on the client side.


> > 2. If a client is disconnected from an Apache WS backend, it will
> connect to
> > another Apache backend automatically and should be able to resume its
> > transaction (for "migratable" sessions), the same as an HTTP translator
> > client.  At this point, I don't remember testing this specifically, but I
> > can't think of any reason it wouldn't work.  Note to self: test this.
> >
>
> Right, it's the migratable session part I was unsure of.  However,
> looking at the code, I don't think the websockets gateway handles
> that.  It looks like it just uses an apache hash object, which (IIUC)
> is process-local.  Am I misreading?
>

You are not misreading.  I was mis-assuming.  A WS session would not be
able to migrate between Apache servers.

The takeaway here is an opensrf session needs to run to completion on
whatever WS server it started on.  If Apache servers are detached and
gracefully shut down, the sessions will be allowed to complete.

-b



> --miker
>
> > The main difference between the HTTP and WS translator is how you
> gracefully
> > remove them from service in a clustered setup.  For HTTP you "detach"
> from
> > the load balancer and give it time for any active requests to finish
> before
> > stopping Apache.  For WS, you detach then issue a graceful Apache stop
> (or
> > restart), which tells the WS translator to kick off clients as soon as
> all
> > active requests are completed.  When all clients are disconnected, the
> > shut-down completes.
> >
> >>
> >> Some background on the technology: even though some browsers don't
> >> support multipart/x-mixed-replace (most, other than IE, do at least
> >> for images -- it's used for simple webcam apps) we were far from the
> >> first to use it, and it's fairly trivial to implement on the client
> >> side.  FWIW, curl supports it, so I wouldn't be too surprised to find
> >> clients in the wild using it.
> >
> >
> > Yes, all good points.  I should have clarified that mixed/replace in
> > combination with XMLHttpRequest is the real issue here.  That
> specifically
> > was only ever supported by FF, and eventually deprecated.
> >
> >>
> >>
> >> All that said, WebSockets is way better and if you write a new
> >> (future) client that uses the translator and multipart/x-mixed-replace
> >> for streaming then you are a bad person and you should feel bad. :)
> >>
> >
> > And pinesol_green will blame you for stealing all the tux dolls.
> >
> > -b
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20160922/ce791bba/attachment-0001.html>