[OPEN-ILS-DEV] login problem with Evergreen and LVS -- SOLVED

[Bill Ott]

Our completely unimpressive franken-cluster: 
http://www.grpl.org/xyz/cluster.jpg
Doug's seen me pull enough cable to know that in the end, it will be 
much prettier.  Probably a blade enclosure with storage arrays.

I've made no secret of my FreeBSD fetish, and since we currently run 30+ 
FreeBSD servers and only 2 Linux servers, we'll use it where we can.

We've done a bit of playing with the import scripts and have a few 
thousand bibs imported.  The test cluster is to give us an idea of how 
things function "split up", and to start giving us an idea of how to 
scale a system.  In the end we'll be looking at about 300K bibs, 600K 
items, and about 100K patrons.

I should probably add a disclaimer that no one can say for certain that 
we will be using Evergreen in the future, but I am a betting man, and 
since they didn't hang me when I took away their MS Office and replaced 
it with OpenOffice...


dkyle said the following on 5/22/2007 5:37 PM:
> Sure.  We are not using VMWare.  We're using pgpool 1 and Postgres on 
> FreeBSD 6.2 boxes, we are replicating - round robin for queries and 
> insert/update copy from primary to secondary.  The EG boxes (real 
> servers in LVS speak) are FC 6 using the arp_announce/arp_ignore 
> sysctl to squelch arp responses for the virtual IP.  The load 
> balancing server is currently LVS on FC 6 using direct route 
> forwarding, and with a single virtual service configured for the 
> virtual IP.
>
> We'll be testing FreeBSD for load balancing also, and setting up 
> mulitple load balance servers with automatic fail over.  Sounds like 
> we have some memcache work to do now too.  It's currently a simple 
> test setup, all on the same subnet - all on the same empty empty desk 
> in the corner.
>
>
>
> Mike Rylander wrote:
>> Mostly out of curiosity (mixed with just a pinch of ego ;), I'd be
>> very interested in hearing more about your setup.  You mentioned
>> pgpool -- are you using pgpool I or II, and are you replicating using
>> that?  And, are you using the VMWare images as the basis of this
>> cluster?  I imagine that answers to those questions would be
>> informative for others as well...
>>
>> TIA
>>
>> --miker
>>
>> On 5/22/07, Mike Rylander <mrylander at gmail.com> wrote:
>>> On 5/22/07, dkyle <dkyle at grpl.org> wrote:
>>> > After some more poking around, I'm guessing that the client 
>>> receives a
>>> > session key from the initial http request, and any subsequent https
>>> > requests to a server that was not aware of that key (the other real
>>> > server in my case) would fail?
>>>
>>> The most likely reason is that the auth service is not using the same
>>> memcache pool on both of the EG app servers.  If this is the case,
>>> you'll likely get errors about not being logged in or sessions timing
>>> out or the like from within the staff client.
>>>
>>> Can you confirm that both EG servers are pointing to the same set of
>>> memcache servers (opensrf.xml =>
>>> /opensrf/default/cache/global/servers/server)?
>>>
>>> --miker
>>>
>>> >
>>> > We had two different virtual services setup, http, and https.  I 
>>> changed
>>> > the LVS to use one virtual service for the virtual IP regardless of
>>> > port, and it works.
>>> >
>>> > dkyle wrote:
>>> > > We are testing a small Evergreen cluster.  We setup two servers
>>> > > running Postgres and pgpool, two "real servers" running 
>>> Evergreen, and
>>> > > one LVS server.  Login with the Evergreen client would sometimes 
>>> fail
>>> > > with a lengthy message regarding no authentication seed being 
>>> found.
>>> > > Some packet capturing revealed that the error occurred whenever the
>>> > > LVS director changed real servers between the clients initial http
>>> > > request, and the subsequent https request.  This seemed a little
>>> > > strange since this involves 2 separate TCP streams.
>>> > >
>>> > > Is this what would be expected? If so, how have others dealt 
>>> with the
>>> > > issue?
>>> > > What is going on with the client login process during the 
>>> initial http
>>> > > request that requires the https request to hit the same real 
>>> server?
>>> > > I did look through the packet decode for that, but I don't 
>>> understand
>>> > > the internal workings enough.
>>> > >
>>> > > Doug.
>>> >
>>> >
>>>
>>>
>>> -- 
>>> Mike Rylander
>>>
>>
>>