<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Nov 4, 2022 at 1:23 PM Mike Rylander <<a href="mailto:mrylander@gmail.com">mrylander@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Bill,<br>
<br>
First, thanks for moving this forward!<br>
<br>
(unfortunately?) Right off the bat, I'm /really/ not liking the idea<br>
of merging application settings (opensrf.xml) into initial connection<br>
settings (opensrf_core.xml). If anything, I'd prefer to see the<br>
application settings move farther away from the file system, somehow,<br>
rather than have to spray that info all over the place -- right now<br>
application config only /has/ to be colocated with the<br>
opensrf.settings app, but it looks like this would require it<br>
everywhere and in potentially only-very-slightly-different files<br>
(differences of service assigned, or service settings, per host) that<br>
would be difficult to troubleshoot.<br></blockquote><div><br></div><div>Not exactly. I didn't dig into this too much in my previous message, but the idea was to support a single file structure, without necessarily requiring a single file.</div><div><br></div><div>A ~/.srfsh.yml file might look like this:</div><div><br></div><div>=====</div><div>message-bus:<br> domains:<br> - name: private.localhost<br> port: 5222<br> credentials:<br> opensrf:<br> username: opensrf<br> password: password<br> connections:<br> srfsh:<br> credentials: opensrf<br> logfile: /openils/var/log/srfsh.log<br> loglevel: debug</div><div>=======</div><div><br></div><div>Gateway, router, standalone client configs could be similarly paired down. Or, if you want just one file (like me), that works too.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Setting, er, application settings aside, though, I /do/ like the idea<br>
of restructuring configuration files to be simpler (or at least more<br>
modern).<br>
<br>
(Ironic historical note: the original config file format was an<br>
enhanced "INI" style, with named section links (even section nesting),<br>
file includes, and typed values -- a lot like yaml today, but without<br>
the whitespace as syntax part.)<br>
<br>
I need to look at your example in more detail and do some more<br>
noodling on the implications, but to answer your question about<br>
"multi-server bricks", yes, we do and will still need that. There are<br>
significant (read: cost-impacting) situations where one needs to run<br>
more horizontally scaled instances of service X than they need for Y,<br>
and in those cases running extra Ys is a waste of resources. I believe<br>
this comes back to "should one 'domain' be able to know about more<br>
than one instance of a service" -- and the answer to that is yes, we<br>
need that. In fact, if you have the concept of a router (even if it's<br>
really just a redis stream now), and you support cross-domain<br>
registration, it seems like it would be extra work to /disallow/<br>
multiple domain-local instances of a service.<br></blockquote><div><br></div><div>OK, good to know. That should be pretty straightforward to add. </div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thanks again for working on this! I hope to have some time to dig into<br>
this in the next week or two.<br></blockquote><div><br></div><div>Awesome, thanks for the feedback!</div><div><br></div><div>-b</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
--<br>
Mike Rylander<br>
Research and Development Manager<br>
Equinox Open Library Initiative<br>
1-877-OPEN-ILS (673-6457)<br>
work: miker@equinoxOLI.org<br>
personal: <a href="mailto:mrylander@gmail.com" target="_blank">mrylander@gmail.com</a><br>
<a href="https://equinoxOLI.org" rel="noreferrer" target="_blank">https://equinoxOLI.org</a><br>
<br>
On Fri, Nov 4, 2022 at 11:40 AM Bill Erickson via Evergreen-dev<br>
<<a href="mailto:evergreen-dev@list.evergreen-ils.org" target="_blank">evergreen-dev@list.evergreen-ils.org</a>> wrote:<br>
><br>
> Hi Devs,<br>
><br>
> Some background:<br>
><br>
> Some may recall my Redis demo at the last EG conference. In the session, I proposed deprecating the OpenSRF router. The discussion continued, particularly with Mike R. and Galen, where we discussed retaining support for multi-domain routing for high availability setups. (Think restarting a service on one domain/brick and having requests to said service get routed to another domain/brick via the Router).<br>
><br>
> Skip ahead, I've started working on code to implement multi-domain support in my Redis branch, but quickly found the existing opensrf_core.xml file to be less than ideal with respect to defining which services run on which domains, among other things.<br>
><br>
> Maybe this is an opportunity to change the configuration file format? As an experiment I put together a sample of what made sense to me:<br>
><br>
> <a href="https://github.com/berick/OpenSRF/blob/user/berick/lpxxx-redisrf-streams-v3/examples/opensrf.yml.example" rel="noreferrer" target="_blank">https://github.com/berick/OpenSRF/blob/user/berick/lpxxx-redisrf-streams-v3/examples/opensrf.yml.example</a><br>
><br>
> [I used YAML because I find it tidy and flexible. I'm more concerned about the configuration structure than the file type, so we could continue using XML or something else, but for my part YAML is superior.]<br>
><br>
> One key difference is that the message bus / connection section is separated into reusable chunks:<br>
><br>
> 1. Routable domains<br>
> 2. Message bus credentials<br>
> 3. Connection types.<br>
> -- These link credentials with logging configs.<br>
><br>
> At runtime, clients/services link a domain with a connection type to derive the bus connection information.<br>
><br>
> Other benefits of the modified structure:<br>
><br>
> * Services are collected into named groups and linked to domains instead of routers.<br>
> * Support for other named configuration chunks. E.g. create a named database setup that can be referenced from the cstore, storage, reporter-store, etc. configurations. Less duplication.<br>
> * One file format for standalone OpenSRF clients, gateways, and services. The file just contains less stuff for clients that don't need the full data set -- or not, either works.<br>
> * Custom connection types for ad-hoc scripts, etc. so they don't require their own config file.<br>
><br>
> And last but not least, the changes work much better for my Redis branch, where we have to be more explicit with our bus domains and their behavior for multi-domain support.<br>
><br>
> One thing not included in the sample are host-specific setups, where a "brick" contains multiple hosts, some running different services than others. This could be added, but I wasn't sure if that was still a common use case.<br>
><br>
> Thoughts? Other config wishlist items?<br>
><br>
> Thanks for reading!<br>
><br>
> -b<br>
><br>
><br>
> _______________________________________________<br>
> Evergreen-dev mailing list<br>
> <a href="mailto:Evergreen-dev@list.evergreen-ils.org" target="_blank">Evergreen-dev@list.evergreen-ils.org</a><br>
> <a href="http://list.evergreen-ils.org/cgi-bin/mailman/listinfo/evergreen-dev" rel="noreferrer" target="_blank">http://list.evergreen-ils.org/cgi-bin/mailman/listinfo/evergreen-dev</a><br>
</blockquote></div></div>