SPAM: Re: [OPEN-ILS-DEV] Full OpenSRF over HTTP

[Mike Rylander]

(NOTE: I'll be putting this up on the wiki soon.)

On 10/18/07, Bill Erickson <erickson at esilibrary.com> wrote:
> Mike Rylander wrote:

[snip]

> > Changes to OpenSRF
> > ==================
> >
> > Stateful communication with backend OpenSRF service instances will be made
> > possible by giving each service the ability to accept "migratable" client
> > remote IDs.  If the configuration section for a given OpenSRF application
> > has the setting
> >
> >    <migratable-clients>true</migratable-clients>
> >
>
> Monty Python aside, how would you feel about "migratory-clients"  (or
> mobile-clients)?  I'm pretty sure "migratable" isn't a word.

Well, "not being a word" has never stopped us before, but I'm not
married to the terminology.  Something to keep in mind is the symmetry
with session and transaction migration on the server side (on the
roadmap),  and (to be a pedant about precision) clients aren't mobile,
nor forced or guaranteed to migrate (implied by "migratory") ... I
guess I'd like something softer -- "client might move, but that's OK."

I consider it an open question at this time.

[snip]

> >
> > Thought, comments, suggestions, considerations?
> >
> >
> Excellent writeup and breakdown of the various pieces...
>

Thanks.

> I have an incomplete, yet, thus far, functionally equivalent prototype
> running on dev.gapines.org.   The example Javascript is at
> http://dev.gapines.org/~erickson/multi/multi.html.  (Warning, produces
> several popups of raw JSON output and only works in Mozilla/FF).  The
> headers are slightly different (X-OSRF instead of X-OpenSRF) and I'm not

At first I had no objection to whatever string was used for the
headers, but I think now that being more explicit is better.  Also, we
need to come up with a convention for all parts of the header.  Your
current code uses "X-OSRF-To" where the proposal says "X-OpenSRF-to".
All else being equal, I feel like the proposal version of these
differences has some more weight to it if only because it is a form of
documentation that can be pointed to, and it's easier to update the
code to match the docs (certainly for things like this) than the other
way around, since the code is more actively worked on.

So ... objections to changing the prototype code's format of the headers to

 X-OpenSRF-<lowercase-descriptive-component>

?

> yet parsing the messages to see when a given request is complete, so
> "collected mode" doesn't function yet, but it's close.
>

For the sake of clarification to those not particularly familiar with
the internals of the existing OpenSRF implementations, this test is
just the transport level stuff -- it's roughly the equivalent of
OpenSRF::Transport::SlimJabber::MessageWrapper.pm -- which is not
trivial in the least, but is only a small part of the stack.  The goal
(well, my goal when writing the proposal) is to enable the entire
OpenSRF stack via an API that's basically identical to the Perl API
(since JS syntax and structure is more similar to Perl in important
ways than, say, C) and very similar to the Python API.  This has the
side effect of making back-end programmers out of any client
programmers -- the API knowledge is directly transferable.

Another thing to note about the prototype is that it only supports
stateless calls right now.  The translator will need to learn (more?)
about opensrf_core.xml in order to construct the correct initial
recipient -- HTTP clients should never know explicitly about XMPP
details, such as the router and domain.  That's covered above in the
discussion of X-OpenSRF-service and X-OpenSRF-to.

I feel like I should say that what I'm pointing out here are not
problems, per se, with the (amazingly quick and very slick) work that
Bill has already done, but simply that the bits in the prototype only
represent a test of some parts of the proposal.

> I think this opens up a lot of possibilities...  One example that comes
> to mind is having the ability to respond with OpenSRF "continue" status
> messages.  This will be great for long-running requests, in particular
> the ILS collections API.  This should help liven up some of the staff
> client displays, as well.  Current practice is to send along sets of IDs
> from the various discovery API's, then go back and fetch the objects 1
> by 1.  With multipart, we can just return a stream of objects and the
> interface can populate rows as the stream arrives, without having to
> "lock up" on a singe batch of objects, all received at once.  I'm
> probably preaching to the choir, here, but I think this is pretty cool
> stuff ;)
>

And the staff client can initiate true transactions in the back-end.
And get actual progress messages from the server Applications (some
already provide them, but they go unused for the most part).

This will open up the staff client to all the varied techniques we
make use of on the server side, and remove the conceptual disconnect
between client programming and server programming that bugs me to no
end ... :)

-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com