[OPEN-ILS-GENERAL] Cucumber for QA

[James Fournie]

Hi there,

One of the suggestions I made with regard to QA in the recent IRC
meeting, was to look into Cucumber for testing.   I thought I'd follow
up a bit.  Caveat that this is not a QA silver bullet, just exploring
some thoughts on it and how it could be used.

Cucumber is a Ruby-based tool for testing, based around
behaviour-driven-development principles.  A key part of Cucumber, is
the Gherkin language is uses.  Gherkin is a business-facing language,
meaning it's designed to be understandable by non-developers.

I've noticed that the Evergreen community has a large range of
technical skill, the documentation group has been one of the most
successful groups in the Evergreen community thanks in part to a lot
of folks who want to contribute but don't have the time or skill to
write code.   In general there are a lot of folks in the community who
don't do much programming but are very technically-minded or systems
thinkers.  I think that Gherkin might be a great way to bridge that
and empower people closer to the end users of the ILS.

 A developer is still required to automate things, but writing the
tests themselves in Cucumber saves some developer effort in coming up
with tests.  It also helps clearly define what features there are and
how they should work, and could be useful for documentation, feature
documentation (ie: for RFPs) and for documenting new development
requirements.

Here's a great blog post which describes the general process:
http://dannorth.net/whats-in-a-story/

Here's an example from that blog post of what the tests would look like:

Scenario 1: Account has sufficient funds
Given the account balance is \$100
 And the card is valid
 And the machine contains enough money
When the Account Holder requests \$20
Then the ATM should dispense \$20
 And the account balance should be \$80
 And the card should be returned

As you can see, it's pretty simple to understand. My colleague Steven
Chan in a Sitka team discussion noted that we do a lot of general
testing in-house with each upgrade and it would be nice to capture
that somehow.  I am sure that other Evergreen implementors have this
knowledge drain as well and I think that Gherkin might be a way to
capture some of that.

I've put together a basic Cucumber Ruby project for testing the TPAC
mainly as a proof of concept.  Look in the 'features' dir at the
'feature' files for the Gherkin:

https://github.com/jamesrf/evergreen-ils-acceptance-test

~James