[OPEN-ILS-DEV] QA proposals

[Galen Charlton]

Hi,

A few concrete proposals for upping our QA game arose from the morning
hackfest.  Here they are for consideration:

Guidelines for patch submission
-------------------------------------------------
[1] Any time a patch adds or alters a stored procedure, pgTAP tests
that exercise its intended functionality should be included.

[2] A change to database or Perl code that fixes a bug should be
accompanied by a Perl (t or live_t) or pgTAP regression test – or by a
statement from the patch author explaining that a test is infeasible
without significant refactoring.

[3] Bugfix patch commit messages should explain how to test the bug it fixes.

For example: rather than just write a bare "LP#124565: fix Evergreen's
cat-petting functionality", provide something more like:

  LP#124565: fix Evergreen's cat-petting functionality

  Evergreen does not do an adequate job of petting cats.  To
  reproduce the problem:

  [1] Bring up an Evergreen OPAC and place a cat
  in front of it.
  [2] Observe that a hand appears and makes waving
  motions a centimeter over the cat.
  [3] Observe that the cat does not purr.
  [4] Apply the patch.
  [5] This time, verify that the hand actually makes contact
  with the cat.
  [6] Expected result: the cat purrs.

To patch authors and testers: please provide feedback and +1/0/-1 on
the three proposed new guidelines for patches.  The first two
essentially flesh out guidelines from
<http://wiki.evergreen-ils.org/doku.php?id=dev:contributing:qa&s[]=pgtap>,
while the third is meant to make it easier for folks to test
complicated patches.

There were two other proposals that folks have offered to work on:

[A] Create a space for sharing manual test cases, such as the ones
that MassLNC and PINES have already stockpiled. Champion: Kathy
Lussier

[B] Institute Test-writing Days, which would be scheduled events for
folks to write automated tests, similar to the the focused Bug
Squashing Days. Champion: Liam Whalen.

Regards,

Galen
-- 
Galen Charlton
Infrastructure and Added Services Manager
Equinox Software, Inc. / The Open Source Experts
email:  gmc at esilibrary.com
direct: +1 770-709-5581
cell:   +1 404-984-4366
skype:  gmcharlt
web:    http://www.esilibrary.com/
Supporting Koha and Evergreen: http://koha-community.org &
http://evergreen-ils.org