[OPEN-ILS-DEV] QA proposals

[Liam Whalen]

I will add to Dan’s concerns.  For large SQL stored procedures, like query_parser_fts, writing a nontrivial test is extremely daunting.  If we apply the requirement to have tests for all changes without consideration of the time necessary to make a valid test, then we risk freezing more complex units of code in their current state.

That being said, as someone in favour of writing tests, it would be simpler to deny all contributions without tests, and this might force the developers’ hands in the area of test writing.  

I am not certain what sort of metric could be used judge the complexity of a piece of code in order to waive the need for tests.  Perhaps, if someone does modify something like query_parser_fts, and there is no test case that can be easily modified to account for the changes, then there could be some sort of petition mechanism to have a test written for the code as it exists in git before the changes that are being proposed.  Then, the developer of the new code, can modify this new test to account for their changes?

Liam Whalen
BC Libraries Cooperative - Sitka
Systems Specialist
855-383-5761 x1022
liam.whalen at bc.libraries.coop

> On Jul 16, 2015, at 12:02 PM, Dan Wells <dbw2 at calvin.edu> wrote:
> 
> Hello all,
> 
> I definitely support requiring tests for *new features*, but I do have some concerns that proposal [2] will have a dampening effect on getting bugs fixed.  I understand the thinking that bug fixes sometimes cause more bugs, but I also think we also need to somehow incentivize bug fixes (being already unglamourous), and this policy makes it that much harder.  I still don't directly oppose the policy, especially since I do not have any great ideas to address this.
> 
> To make this more than abstract, we at Calvin have perhaps 10 or 20 smallish bug fixes in our local repository.  It is a constant struggle to justify to our staff the time it is going to take me to get these into LaunchPad.  If we're expected to also write tests, the timeline for getting these into LP probably just went from within 6 months to within 2 years, and maybe that is too generous.
> 
> So, if we are adopting [2], I think we need to be careful to not marginalize or otherwise discourage sharing branches without tests, as that still gets us one step closer to where we need to be.  Ideally we might find some people willing to specialize in test writing, and perhaps someday our tests will be so comprehensive that many bug fixes won't need a totally new test, but neither outcome seems imminent at this time.
> 
> Sincerely,
> Dan
> 
> ________________________________________
> From: Open-ils-dev <open-ils-dev-bounces at list.georgialibraries.org> on behalf of Galen Charlton <gmc at esilibrary.com>
> Sent: Wednesday, July 15, 2015 12:25 PM
> To: Evergreen Development Discussion List
> Subject: Re: [OPEN-ILS-DEV] QA proposals
> 
> Hi,
> 
> On Wed, Jul 15, 2015 at 12:18 PM, Kathy Lussier <klussier at masslnc.org> wrote:
>> Are these guidelines official now?
> 
> As no objections were expressed... they are now.
> 
>> If so, I would like to update some wiki pages to reflect the new
>> requirements:
>> 
>> http://wiki.evergreen-ils.org/doku.php?id=contributing
>> http://wiki.evergreen-ils.org/doku.php?id=dev:signoff_review_checklist
> 
> Thanks!
> 
> Regards,
> 
> Galen
> --
> Galen Charlton
> Infrastructure and Added Services Manager
> Equinox Software, Inc. / The Open Source Experts
> email:  gmc at esilibrary.com
> direct: +1 770-709-5581
> cell:   +1 404-984-4366
> skype:  gmcharlt
> web:    http://www.esilibrary.com/
> Supporting Koha and Evergreen: http://koha-community.org &
> http://evergreen-ils.org