[OPEN-ILS-GENERAL] Evergreen & Software Performance Analysis

[Elfstrand, Stephen F]

You're right Mike. "direct calls to the database" is incorrect usage. And we are all grateful to the developer community for their great work so far.  So please take my suggestions for improvement in the context of this gratitude. I want  Evergreen to get to the next level.

My point still stands that features have been released that are way too slow (Acquisitions: displaying POs & invoiceswith many items) or just don't work (Authorities).  What is the point of releasing them?

If "premature optimization is the root of all evil"  how should we describe less than great performance in a  production release? Yes we want to test at scale, that's what Alpha and Beta releases are for. We need a large-ish site to install the Alpha and Beta releases (without significant customizations or features borrowed from future or past releases) and run them on production, do the QA optimization where problems appear, document the testing (and publish it) all before the production release.

If the release manager can't say no, where then should the QA decision be made?  QA should be part of the release process somewhere and contracts for development should include a QA clause and performance measures. They should also require adequate documentation. The delay of a release because of QA problems or lack of documentation is a motivator to do the QA & documentation; without that they have been considered of secondary importance.

You say that "RM role is a single volunteer position with no real power except to say "no" to a release. Can't they say no to just a feature or process if it doesn't measure up? Is it really just an all or nothing situation?

I think that a production release should be a complete package with QA, complete documentation and comprehensive, reliable upgrade scripts, not just a bunch of features of uneven quality. That just feeds into the "Evergreen (and Open Source in general) is not good enough for large libraries" myth.

Personally, I'd rather see one great release a year than two per year that have issues along with the improvements.

Stephen


From: Mike Rylander [mailto:mrylander at gmail.com]
Sent: Thursday, February 28, 2013 10:47 AM
To: Evergreen Discussion Group
Cc: Elfstrand, Stephen F
Subject: Re: [OPEN-ILS-GENERAL] Evergreen & Software Performance Analysis



On Thu, Feb 28, 2013 at 10:56 AM, Jason Stephenson <jstephenson at mvlc.org<mailto:jstephenson at mvlc.org>> wrote:
Quoting "Elfstrand, Stephen F" <stephen.elfstrand at mnsu.edu<mailto:stephen.elfstrand at mnsu.edu>>:
there are a lot of direct calls to the database from the client,

Speaking as one of the developers, I have to correct the above statement. No such thing happens in Evergreen. The client cannot talk directly to the database, it goes through OpenSRF which handles database communication on the backend.

And speaking as another of the developers, I'd love to see evidence of problems (really -- we can't address issues without it).  New features and their supporting code and database structures certainly introduce new opportunities for optimization (database indexes, condensed APIs, etc), but these opportunities are often not visible until real-world load is applied, and in the words of a great man, "premature optimization is the root of all evil" (http://c2.com/cgi/wiki?PrematureOptimization).  If you have such evidence, please share.  I, for one, have a nice pile of evidence and addressing changes that are becoming a working branch right now.  As for assigning the job of QA to the release manager, assuming the community wants a 6-month release cycle and admitting that the RM role is a single volunteer position with no real power except to say "no" to a release, I don't think that is reasonable as the structure exists today.

In the larger sense, I believe we're driving at the same point, of course -- the lack of QA, particularly performance QA, before a release.  There is a chicken-and-egg problem here, however, and what we really need, as a community, is an ongoing effort -- certainly supported by and augmented with specific tactical audits of all the things you point out, and more -- that is separate from the release process itself (though informing it, obviously) to make sure that the currently desired round of  analysis and repair is not the last.  It's that ongoing effort, and the human and technical infrastructure that must, somehow, be sustained that I think is a bigger problem than any individual performance regression.

Equinox is making the attempt at creating that.  We've been getting good, positive feedback, and our efforts are intentionally orthogonal to, and /not/ mutually exclusive with, all other the discussions currently ongoing.  We're working hard to make sure that stays true, and that what we end up doing with and for the rest of the community is part of a larger whole.  We don't want to waste anyone's time, so we're making sure we get things as right as possible out of the gate, but we'll be sharing more information soon.  However, and I can not stress this enough, this should not cause anyone to stop their own efforts!  We all benefit from as much participation as possible by all types of community members: service providers, libraries and those institutions that straddle the line between both.

--
Mike Rylander
 | Director of Research and Development
 | Equinox Software, Inc. / Your Library's Guide to Open Source
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com<mailto:miker at esilibrary.com>
 | web:  http://www.esilibrary.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-general/attachments/20130322/dd9c7579/attachment-0001.htm>