[OPEN-ILS-DEV] Evergreen & Software Performance Analysis

[Kathy Lussier]

Hi all,

I wasn't sure if I should add this to the QA discussion, but it seemed 
worthy of its own thread.

During the "future of the staff client" meeting, I advocated for 
bringing in a consultant to do a software performance analysis for 
Evergreen to help us identify where the critical bottlenecks are in the 
system in the hopes that we could then identify the areas that need to 
be worked on to improve performance. At the time, I didn't have any 
concrete suggestions on finding a consultant who could take on this 
project, but I have since done some more investigation and have a couple 
of leads, the most promising of which is an individual local to 
Massachusetts who previously worked for many years at Stratus 
Technologies where he was involved in all levels of performance 
analysis. He now teaches graduate-level courses on performance 
evaluation and also does contract work.

Now that I actually have concrete leads, I would like to get the ball 
rolling, provided there is support from the larger community. I'm not 
quite sure how this might fit in with ESI's planned QA efforts or with 
the possibility of bringing in a firm like OmniTI as Dan suggested, but 
my reading into these QA e-mails is that the focus would be on testing 
new commits.  I fully support improved testing for new code, but my main 
concern now is testing what is already there since improving existing 
performance issues is a high priority for us. I think there is 
opportunity for these efforts to support one another.

As I mentioned during the meeting, MassLNC is willing to contribute 
financially to bringing on a consultant, but I think it's important that 
this effort is a community project for several reasons. If we do bring a 
consultant, in order to be successful, it is imperative that they work 
with the community of sys admins and developers who already have insight 
on where the underlying issues might be. Also, if we don't have buy-in 
from the community, then it is likely that any work a consultant does 
will be largely ignored. My hope is that any work done by a 
consultant(s) could be a real starting point in resolving any of the 
critical bottlenecks that were identified in a performance evaluation.

So I guess my question is whether there is any support from the 
community to proceed with finding someone to do a software performance 
analysis?

If the answer is yes, then there are at least two things that need to be 
done before going any further.

1. We need to define the areas that need analysis. Bill Erickson had 
suggested during the meeting that we might find a lot of areas of 
agreement if we worked on our own list of ideas before contracting with 
anyone. The consultant I spoke with also suggested that we identify the 
areas where the pain is being most felt. If there is support for moving 
forward, perhaps we can begin to build this list on the wiki. As a side 
note, the consultant also recommended pulling together a group of 
stakeholders to oversee this process. I initially was thinking this 
group could just be the existing developer community, but, with the 
recent flurry of discussion over QA and performance, I'm now wondering 
if it might be worthwhile to have a QA group that focuses on 
coordinating these efforts and identifying next steps.

2. We also would need to identify an environment to do the performance 
testing. In speaking with the performance consultant, the ideal testing 
environment would be a server that has a tool that drives transactions. 
Is that something that testing.esilibrary.com has? If not, is it 
something that could be built. Can Cucumber help us with that?

I appreciate any thoughts you all have on this subject.

Kathy

-- 
Kathy Lussier
Project Coordinator
Massachusetts Library Network Cooperative
(508) 343-0128
klussier at masslnc.org
Twitter: http://www.twitter.com/kmlussier

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20130220/b7b29c0d/attachment.htm>