[OPEN-ILS-DEV] QA proposals

Fri Jul 17 10:02:56 EDT 2015

Hi Dan and Liam,

Thanks for sharing your concerns regarding the time investment required 
to meet the QA requirements as proposed. I do understand that our 
developer resources in the community are low and time is at a premium, 
and I think we need to be aware of these constraints as we move forward 
with any QA requirements.

I do want to point out that item 2 had a caveat at the end - "or by a 
statement from the patch author explaining that a test is infeasible 
without significant refactoring." Does the inclusion of this caveat help 
to address some of those concerns?

Although the idea of a petition mechanism might work in larger 
communities, for our community, it leaves me with the question of who 
would have the time to then write that test is some kind of petition 
were made.

I do want to respond to this comment from Dan:

> Ideally we might find some people willing to specialize in test writing, and perhaps someday our tests will be so comprehensive that many bug fixes won't need a totally new test, but neither outcome seems imminent at this time.

During the January developers meeting, where we had the QA discussion 
that led to the new requirements, I posed the question of whether we 
needed to explore the idea of bringing on a person with an explicit 
commitment to review patches and write tests. The response I received to 
the idea was somewhat lukewarm, for reasons that I totally understand. 
However, if the developer community told me that they really needed 
somebody on board who specialized in test writing to be able to do the 
type of QA recommended in the Equinox report, I would do whatever I 
could to try to get the funding and expertise to make that happen.

There were several Evergreen sites that felt strongly enough about QA to 
put funds into the Equinox QA study. I might be an optimist, but I 
believe we could get similar support in bringing this type of expertise 
on board.

I would love to see the Calvin bug fixes in Launchpad sooner rather than 
later. If the time constraints on doing these tests is indeed too high 
for the developer time we have in the community, then we're left with a) 
lowering the bar for QA or b) doing what we can to grow the community so 
that we have somebody who can make the commitment to that work. I'm 
inclined to go with b) because I think it will lead to overall 
improvement for the software our libraries rely on.

I'll also note that, at the same developers meeting, I volunteered to do 
an environmental scan of other projects to see how they handled these 
issues. It's something I have not done, other than some initial 
brainstorming with Galen, partially because I was waiting to see how 
things progressed with the new QA requirements and the test writing 
days. However, I'm willing to re-focus my energies there if people think 
it's useful.

Kathy

On 07/16/2015 03:02 PM, Dan Wells wrote:
> Hello all,
>
> I definitely support requiring tests for *new features*, but I do have some concerns that proposal [2] will have a dampening effect on getting bugs fixed.  I understand the thinking that bug fixes sometimes cause more bugs, but I also think we also need to somehow incentivize bug fixes (being already unglamourous), and this policy makes it that much harder.  I still don't directly oppose the policy, especially since I do not have any great ideas to address this.
>
> To make this more than abstract, we at Calvin have perhaps 10 or 20 smallish bug fixes in our local repository.  It is a constant struggle to justify to our staff the time it is going to take me to get these into LaunchPad.  If we're expected to also write tests, the timeline for getting these into LP probably just went from within 6 months to within 2 years, and maybe that is too generous.
>
> So, if we are adopting [2], I think we need to be careful to not marginalize or otherwise discourage sharing branches without tests, as that still gets us one step closer to where we need to be.  Ideally we might find some people willing to specialize in test writing, and perhaps someday our tests will be so comprehensive that many bug fixes won't need a totally new test, but neither outcome seems imminent at this time.
>
> Sincerely,
> Dan
>
> ________________________________________
> From: Open-ils-dev<open-ils-dev-bounces at list.georgialibraries.org>  on behalf of Galen Charlton<gmc at esilibrary.com>
> Sent: Wednesday, July 15, 2015 12:25 PM
> To: Evergreen Development Discussion List
> Subject: Re: [OPEN-ILS-DEV] QA proposals
>
> Hi,
>
> On Wed, Jul 15, 2015 at 12:18 PM, Kathy Lussier<klussier at masslnc.org>  wrote:
>> Are these guidelines official now?
> As no objections were expressed... they are now.
>
>> If so, I would like to update some wiki pages to reflect the new
>> requirements:
>>
>> http://wiki.evergreen-ils.org/doku.php?id=contributing
>> http://wiki.evergreen-ils.org/doku.php?id=dev:signoff_review_checklist
> Thanks!
>
> Regards,
>
> Galen
> --
> Galen Charlton
> Infrastructure and Added Services Manager
> Equinox Software, Inc. / The Open Source Experts
> email:gmc at esilibrary.com
> direct: +1 770-709-5581
> cell:   +1 404-984-4366
> skype:  gmcharlt
> web:http://www.esilibrary.com/
> Supporting Koha and Evergreen:http://koha-community.org  &
> http://evergreen-ils.org

-- 
Kathy Lussier
Project Coordinator
Massachusetts Library Network Cooperative
(508) 343-0128
klussier at masslnc.org
Twitter:http://www.twitter.com/kmlussier