[OPEN-ILS-GENERAL] Cucumber for QA

[James Fournie]

Hi Rogan,

I think a server isn't even 100% necessary, as long as you have a
running EG instance somewhere to point it at you can run it on a OS X
or Linux workstation.  In fact, since Selenium requires a browser, you
may have better results that way.  That said, I constructed my demo to
optionally use PhantomJS instead, which could be run on a server
without X11.  Aside from Selenium or PhantomJS, the main requirement
of my demo is Ruby plus some Ruby libraries which may not necessarily
be packaged by older distros.

Though to reiterate some very important points that Dan made, I don't
know if it's the best to get bogged down in tools too much.  I regret
posting as I think I've over-emphasized Cucumber itself.

A good example is a conversation I had today with my colleague Tina,
we are looking at improving the org unit hiding behaviour for Sitka's
needs.  We found that editing holds could be a problem -- in some
cases patrons could edit a hold's pickup location to something that
would never fulfill.   Tina and I had some back and forth via email in
which I picked at the code  to see if the behaviour was as desired,
and which resulted in a long email thread.  I synthesized a small part
of our discussion into the following Gherkin:

Scenario: Holds placement range
    Given I am in the Whistler TPAC
    And Whistler's org hiding depth is set to 2
    When I place a hold on an item
    Then I should only be able to select Whistler as my pickup location

Scenario: Holds pickup editing range
    Given I am in the Whistler TPAC
    And Whistler's org hiding depth is set to 2
    And I have a hold at Kamloops
    And I have a hold at Whistler
    When I edit my hold at Kamloops
    Then I should only see Thompson-Nicola Regional District Library
branches in the pickup location editor
    When I edit my hold at Whistler
    Then I should only see Whistler in the pickup location editor

In this example I've used Sitka's libraries, you could substitute the
generic example branches or your own library's branches.  And this use
case is probably not for everyone, but I think you get a sense of it.

Just recording this kind of information is useful.  Even without
automation, I can follow those instructions to reproduce the test.
It's something that less-technical staff could record and use for
manual testing.

Taking it to the next level, non-technical administrators can easily
use this framework to document their requirements of a new feature and
how it should behave.  The developer then knows what they're writing
and is better able to meet those requirements..

Quality can be defined many ways, but one way is "meeting requirements
and expectations".  This means the community needs to define
requirements and expectations and then ensure Evergreen meets them.
If you are writing any tests, this will probably naturally happen,
Cucumber is just a way that non-developers can lead the charge.

Another key here is process --  integrating some sort of consistent QA
process in the development work flow.  There is a QA process now --
developers must sign off their commits and have someone else sign it
off.  But I think many are in agreement that more than that is needed
in the form of a clearly defined testing process beyond just "test
your code".   It's difficult because everyone has their own idea about
what kind of testing should happen and the tools and testing methods
are going to vary a lot depending on what part of the stack you're
working on.

~James