[OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

[Mike Rylander]

Dan,

You're right, and I was personally remiss in leaving out a pgTAP test
for my recent repairs to authority overlay generation in an attempt to
avoid delaying 2.5 even a little bit.  I've pushed
120e24dd6d26f2460d209421bbbc9a1777f4fb52 in order to remedy that.  (Of
course, if I got anything wrong there, let me know and I'll try to fix
it.)  Over the next few days I'll look back at other recent DB commits
to see how they could be tested as well.

Thanks for trying to keep us all honest.

--miker

On Fri, Nov 1, 2013 at 5:01 PM, Dan Scott <dan at coffeecode.net> wrote:
> In the QA report for which several members of the community generously
> paid $30,000, the "Moving Forward" section at
> http://nox.esilibrary.com/~jason/qareport/qa.html#_moving_forward
> states:
>
> "We recommend that the development community start including integration
> tests with their changes to the backend, and pgTAP tests with their
> database changes (there was discussion and general interest in this
> during a developers meeting)."
>
> The referenced developer meeting is minuted at
> http://evergreen-ils.org/meetings/evergreen/2013/evergreen.2013-08-27-14.04.html
> with the agreement "general interest in easing into qa practices
> demonstrated by phasefx, phasefx to hold hands with everyone, especially
> through the mailing list"
>
> I'm concerned that since then we have seen a number of database changes
> without corresponding pgTAP tests (the most recent pgTAP test was
> committed Sept 3rd, while there were a ton of changes committed through
> October). I did bring this up in the IRC channel with respect to one of
> the recent bug fixes that was committed for a fairly fundamental
> function, but my gentle prod in that case appears to have been
> overlooked.
>
> I recognize there is pressure to get the 2.5 release out, but if we
> continue to follow our past approach of not including unit tests where
> the path has been blazed for us when we commit changes to the database,
> our quality assurance is going to assuredly be similar to the quality we
> have produced in the past.
>
> In short, we have heard strongly from the community that we are not
> producing software of the quality that they expect; and the community
> has followed up those words with a significant investment in the QA
> project; and we as a developer team are _not_ following through with the
> process improvements we had agreed to adopt.
>
> FWIW, should anyone want to follow some commits to teach themselves how
> to add pgTap tests, I did go through the learning process to support
> https://bugs.launchpad.net/evergreen/+bug/1242999 - the tests I created
> and the instructions for running them are basic, but they're a start.
> Thanks to Galen for giving me a pointer in the right direction.



-- 
Mike Rylander
 | Director of Research and Development
 | Equinox Software, Inc. / Your Library's Guide to Open Source
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com