[OPEN-ILS-DEV] Putting the community's QA money where our dev mouth is

[Dan Scott]

On Tue, Nov 5, 2013 at 2:38 PM, Mike Rylander <mrylander at gmail.com> wrote:
> Dan,
>
> I understand the process of building mock environments, but thanks.

You're welcome. I'm sure that many other people reading this thread,
now and in the future, might not, so hopefully my response will be
helpful to them in understanding what we're talking about.

> It's my understanding that concerto is intended to be a valid test
> environment, so I decided to go with that.  Is that not the case?

As I mentioned in IRC, I have experience with a previous project where
a sample database meant to demonstrate the usage of a product had
regression tests depend on its structure over time, so much so that
changing the sample database to use better examples and to show off
new features became out of the question. I would prefer to avoid a
similar fate with this project.

I think the Concerto sample database is useful for demonstrating
Evergreen and for users to report problems. I'm not, however, a big
fan of building dependencies on it for testing purposes.

Building a test that relies on Concerto also means that once you log
into the system and make any changes that affect the database, you can
no longer depend on the results of tests run against that database.
You need a full stop, database rebuild, start of the system to get a
valid set of test results again. And anything that slows down a
developer is less likely to actually happen, right?

> As I mentioned in IRC, changing my most recent test to create a mock
> env would be fine, of course, and might even be a good learning tool
> for someone else interested in working with pgTAP tests in EG.  But,
> if nobody else gets around to it, I'll try to find time to remove the
> dependency on concerto data.  It'll be lower priority than other tests
> or new development, though, since the test exists now ... and that's
> 100% more existing than it was doing before. :)

It's hard to argue with that.