[OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

[Mike Rylander]

CORRECTION: the commit hash is
3e7f6f1503938e3e5a1cfe3bd90eaa8f021a0d48 ... I failed to fetch before
attempting to pick. Sorry!

--miker

On Tue, Nov 5, 2013 at 12:41 PM, Mike Rylander <mrylander at gmail.com> wrote:
> Dan,
>
> You're right, and I was personally remiss in leaving out a pgTAP test
> for my recent repairs to authority overlay generation in an attempt to
> avoid delaying 2.5 even a little bit.  I've pushed
> 120e24dd6d26f2460d209421bbbc9a1777f4fb52 in order to remedy that.  (Of
> course, if I got anything wrong there, let me know and I'll try to fix
> it.)  Over the next few days I'll look back at other recent DB commits
> to see how they could be tested as well.
>
> Thanks for trying to keep us all honest.
>
> --miker
>
> On Fri, Nov 1, 2013 at 5:01 PM, Dan Scott <dan at coffeecode.net> wrote:
>> In the QA report for which several members of the community generously
>> paid $30,000, the "Moving Forward" section at
>> http://nox.esilibrary.com/~jason/qareport/qa.html#_moving_forward
>> states:
>>
>> "We recommend that the development community start including integration
>> tests with their changes to the backend, and pgTAP tests with their
>> database changes (there was discussion and general interest in this
>> during a developers meeting)."
>>
>> The referenced developer meeting is minuted at
>> http://evergreen-ils.org/meetings/evergreen/2013/evergreen.2013-08-27-14.04.html
>> with the agreement "general interest in easing into qa practices
>> demonstrated by phasefx, phasefx to hold hands with everyone, especially
>> through the mailing list"
>>
>> I'm concerned that since then we have seen a number of database changes
>> without corresponding pgTAP tests (the most recent pgTAP test was
>> committed Sept 3rd, while there were a ton of changes committed through
>> October). I did bring this up in the IRC channel with respect to one of
>> the recent bug fixes that was committed for a fairly fundamental
>> function, but my gentle prod in that case appears to have been
>> overlooked.
>>
>> I recognize there is pressure to get the 2.5 release out, but if we
>> continue to follow our past approach of not including unit tests where
>> the path has been blazed for us when we commit changes to the database,
>> our quality assurance is going to assuredly be similar to the quality we
>> have produced in the past.
>>
>> In short, we have heard strongly from the community that we are not
>> producing software of the quality that they expect; and the community
>> has followed up those words with a significant investment in the QA
>> project; and we as a developer team are _not_ following through with the
>> process improvements we had agreed to adopt.
>>
>> FWIW, should anyone want to follow some commits to teach themselves how
>> to add pgTap tests, I did go through the learning process to support
>> https://bugs.launchpad.net/evergreen/+bug/1242999 - the tests I created
>> and the instructions for running them are basic, but they're a start.
>> Thanks to Galen for giving me a pointer in the right direction.
>
>
>
> --
> Mike Rylander
>  | Director of Research and Development
>  | Equinox Software, Inc. / Your Library's Guide to Open Source
>  | phone:  1-877-OPEN-ILS (673-6457)
>  | email:  miker at esilibrary.com
>  | web:  http://www.esilibrary.com



-- 
Mike Rylander
 | Director of Research and Development
 | Equinox Software, Inc. / Your Library's Guide to Open Source
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com