[Evergreen-dev] Thoughts on experimental OpenSRF config format change

[Mike Rylander]

On Fri, Nov 4, 2022 at 1:47 PM Bill Erickson <berickxx at gmail.com> wrote:
>
> On Fri, Nov 4, 2022 at 1:23 PM Mike Rylander <mrylander at gmail.com> wrote:
>>
>> Hi Bill,
>>
>> First, thanks for moving this forward!
>>
>> (unfortunately?) Right off the bat, I'm /really/ not liking the idea
>> of merging application settings (opensrf.xml) into initial connection
>> settings (opensrf_core.xml).  If anything, I'd prefer to see the
>> application settings move farther away from the file system, somehow,
>> rather than have to spray that info all over the place -- right now
>> application config only /has/ to be colocated with the
>> opensrf.settings app, but it looks like this would require it
>> everywhere and in potentially only-very-slightly-different files
>> (differences of service assigned, or service settings, per host) that
>> would be difficult to troubleshoot.
>
>
> Not exactly.  I didn't dig into this too much in my previous message, but the idea was to support a single file structure, without necessarily requiring a single file.
>

Hrm... maybe I'm misunderstanding, sorry if that's the case.  Let me
try to explain my thought process, concerns, and desires.

After my first reading of your original message, I thought you were
basically only talking about yaml-izing opensrf_core.xml.  I'm fully
on board with making that smarter (reusable chunks, etc) and like you
I don't care much what the format is in the long run.  And, FTR, I'm
fully on board with making opensrf.xml smarter, or more dynamic, or
non-existent (as a filesystem object).

But then I saw Blake's response that mentioned service defaults, and
looked at your linked example more closely, and that's when I came
away with the impression that you may /also/ be wanting to embed the
application config (what opensrf.settings reads and then distributes
to applications) in the connection config file /for the services to
look at directly/.  Is that correct?

My fervent hope is that we retain the separation of concerns that the
two files embody today.  Right now, we just tell things how to connect
to the opensrf network via opensrf_core.xml, and then they ask the
opensrf.settings service for their config data.  I promise that's not
just an idle desire!  Because there are two "layers" (for lack of a
better term) of configuration, and human error is unfortunately still
a thing, it increases the risk that things will become broken in a
complicated environment when All The Things live in just one place,
instead of dedicated (and tool-enrobed) locations for different
configuration concerns and targets.

To expand a little on my mention of opensrf.xml above, eventually I
would like to see the opensrf.settings service stop getting it's
pile-o-data from opensrf.xml (or any file, for that matter) and
instead read it from a structured data store.  Maybe a full-on
database, maybe a simple persistent document store, but something more
machine-actionable than a file that a human edits and SCP's around.

Put another way, I don't want services to read a local file for
anything other than initial bootstrapping. If we do without even a
bootstrap config file and just have command line parameters, all the
better.  I don't see a way to do that right now, though.

BUT WAIT!

That all looks pretty negative, so let me offer a counter to myself.

If what you meant (and what I misunderstood) was that opensrf.xml
could/would eventually turn into a yaml file which would have mutually
exclusive structural sections from opensrf_core.yml, and
opensrf.settings could be /actively/ told to read a yaml file that
just happens to be the same file on disk for both bootstrap and
settings data purposes  (or, I guess, be pointed to a section?) --
meaning, opensrf.settings would just look at different parts of the
file for each purpose and other services would just ignore all of the
non-bootstrap parts, and not try to use them directly without going to
the settings service -- then I retract my concerns.

Thanks, again!

--Mike