[Evergreen-dev] Thoughts on experimental OpenSRF config format change

[Bill Erickson]

Hi Devs,

Some background:

Some may recall my Redis demo at the last EG conference.  In the session, I
proposed deprecating the OpenSRF router.  The discussion continued,
particularly with Mike R. and Galen, where we discussed retaining support
for multi-domain routing for high availability setups.  (Think restarting a
service on one domain/brick and having requests to said service get routed
to another domain/brick via the Router).

Skip ahead, I've started working on code to implement multi-domain support
in my Redis branch, but quickly found the existing opensrf_core.xml file to
be less than ideal with respect to defining which services run on which
domains, among other things.

Maybe this is an opportunity to change the configuration file format?  As
an experiment I put together a sample of what made sense to me:

https://github.com/berick/OpenSRF/blob/user/berick/lpxxx-redisrf-streams-v3/examples/opensrf.yml.example

[I used YAML because I find it tidy and flexible.  I'm more concerned about
the configuration structure than the file type, so we could continue using
XML or something else, but for my part YAML is superior.]

One key difference is that the message bus / connection section is
separated into reusable chunks:

1. Routable domains
2. Message bus credentials
3. Connection types.
  -- These link credentials with logging configs.

At runtime, clients/services link a domain with a connection type to derive
the bus connection information.

Other benefits of the modified structure:

* Services are collected into named groups and linked to domains instead of
routers.
* Support for other named configuration chunks.  E.g. create a named
database setup that can be referenced from the cstore, storage,
reporter-store, etc. configurations.  Less duplication.
* One file format for standalone OpenSRF clients, gateways, and services.
The file just contains less stuff for clients that don't need the full data
set -- or not, either works.
* Custom connection types for ad-hoc scripts, etc. so they don't require
their own config file.

And last but not least, the changes work much better for my Redis branch,
where we have to be more explicit with our bus domains and their behavior
for multi-domain support.

One thing not included in the sample are host-specific setups, where a
"brick" contains multiple hosts, some running different services than
others.  This could be added, but I wasn't sure if that was still a common
use case.

Thoughts?  Other config wishlist items?

Thanks for reading!

-b
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://list.evergreen-ils.org/pipermail/evergreen-dev/attachments/20221104/3a2c260b/attachment.htm>