[OPEN-ILS-DEV] Rethinking the release process: (was: Trac usage, EG Versions)

[Dan Scott]

On Fri, 2009-09-11 at 20:24 -0400, Joe Atzberger wrote:
> I'm looking to formally record in Trac the various bugs encountered
> during development and testing.  Right now, the number of EG Versions
> to file against are a small subset of the actual number of available
> versions (plus 2.0 that doesn't yet exist).  So I have a couple
> questions:
>       * How should the 2.0 Version should be used?  
>       * What level of granularity do we want: each increment,
>         major.minor only, something else?  
>       * What about trunk?
> I suggest we probably want each increment, but more important than
> that, I think we need a version for trunk (or HEAD) where the majority
> of bugs and enhancements will be filed... and some hopefully fixed.
> It would be counterproductive to file enhancements against a stable
> version, since all the new features should be going against trunk when
> possible.  And you don't want it against whatever number version
> happens to be current now, because if the ticket sits a while or the
> feature takes more time to develop, it needs to keep up w/ the
> progressing version numbers and not look like a piece of history.
> 
> Any thoughts on this?

Hey Joe:

A simple reply would be that one can and should modify tickets if a
feature targeted for a given release gets bumped to a subsequent
release.

However, I think it would be worth revisiting how releases work in
general; this is just one aspect (but an important one, I would love to
see more formal use of Trac for bugs and features and I think it would
be very healthy for the community).

Our development and release process is a work in progress, as
http://open-ils.org/documentation/contributing.html states. I'm going to
suggest a few changes of direction that might help us move towards a
more agile release process (and hopefully get a conversation going about
other changes that we might want to consider).

* Time-based releases: We've seen projected release dates slip because
we were trying to fit a feature into a given release and waiting on that
feature to be finalized. Meanwhile, other enhancements get stuck waiting
on that release. Generating a regular (monthly? weekly? daily?) release
tarball would give us the opportunity to have regular, objective
feedback on the quality of our releases, get new features into the hands
of interested parties with less hassle, and still give us the ability to
anoint a given tarball an official release. But this won't be possible
without a few other wishlist items:

* Automated releases: Right now, the effort required to cut a release
comes down to a whole lot of (rather poorly documented) manual work that
doesn't really encourage a "release often" approach. If we were to
automate the process of creating a release tarball & corresponding staff
client, we could start pumping out time-based releases that could get
new features into the hands of testers or adopters earlier. This would
require us to be disciplined about maintaining a database upgrade script
each time the database schema is modified, rather than the heroic effort
that miker has put into each and every database upgrade script to date.
We would find out right away if fundamental pieces of the build break
due to dependencies (such as the translations, which frustrated miker a
while back when some of the Python modules used by the build were
updated and broke our existing translation build scripts).

* Automated testing: This is one area where Evergreen, a complex beast,
is currently frighteningly weak - but I hear that there's some work
underway to build a sample database, which would be a great base for a
suite of regression tests (at least at the OpenSRF level; automated
testing of XUL / Dojo interfaces / catalogue is another more complex
kettle of fish, but it has been done by others...) Once we have that
base available to us, I think it should be a condition that each new
feature should come with a suite of tests to ensure that changes in
another part of the stack (whether our code changes, or changes to
external dependencies - hello, PostgreSQL 8.4!) don't unintentionally
affect how this particular feature works. There's loads of room for
adding test coverage of existing functionality, of course. When a bug is
found, a test should be added that triggers the bug (and when the bug is
fixed, the test will start passing).

* Divide and conquer responsibilities: I think there are a number of
responsibilities that could be picked up by more members of the
community. Sharing responsibilities across a broader team should make
everyone more efficient (at the cost of some communication and
coordination overhead). Some ideas about possible roles:
  * Translation management, currently rather poorly done by me, could be
relatively easily picked up by someone else (particularly if Pootle is
adopted as an official tool). 
  * Platform owners - people responsible for testing Evergreen releases
on a particular distribution & maintaining installation documentation /
install scripts for that distribution & dependency Makefiles & possibly
generating virtual images. Currently there's a rough focus on Debian and
Ubuntu, but there are members of the community who could cover RHEL, and
we could have better defined coverage for specific versions of Debian
and Ubuntu.
  * Tests: once we have a test framework and some good sample tests in
place, I'm sure there are a number of community members who could start
writing tests for existing functionality. For that matter, perhaps there
are community members who can help build that framework.
  * Patch management: I've seen a few contributed patches get sent to
the list with little feedback from those of us with commit bits. Some
other projects more formally track the status of patches to ensure that
a given patch gets reviewed and that it is either accepted, rejected, or
that feedback is given and that a revised patch has been requested -
either via a wiki page, or via a regular list of patches sent to the
development mailing list. Someone, or someones, could be responsible for
tracking the disposition of all patches (and helping us ensure that we
give feedback to would-be contributors in a timely fashion).

On another track, versioning, right now I think there are three or four
active release branches:

1.2 - hopefully limited to just bugfixes (but, last I heard, there were
still a lot of sites on 1.2)
1.4 - hopefully limited to just bugfixes (now that we have a usable
release in 1.4.0.6, by all accounts)
1.6 - holding tank for a ton of new features, with acquisitions and
serials and the event-driven infrastructure
2.0 currently aka trunk - holding tank for everything else in
development

Each release since 1.2 has an a.b.c.d numbering scheme, which doesn't
jive with our documented versioning scheme
(http://evergreen-ils.org/dokuwiki/doku.php?id=versioning). Going
forward, I suggest we toss the a.b.c.d scheme that crept in with the 1.2
release and stick with the documented approach of features only going
into the "a" or "b" iterations and bugfixes only going into the "c"
iterations. Also, perhaps we should just use increments of 1, rather
than suggesting that an odd release number is considered unstable. We
haven't pushed out an odd release since 1.1 (May 2007), so we're not
really making use of this distinction. If we went with time-based
releases, then these could be date-stamped tags in SVN, and we could
just tag the one that we pick for the next official release as a.b+1.0.

We could also adopt a support policy of something like "current a
release + previous a release" (picking numbers out of thin air, assuming
that we did something like bump "a" once a year and "b" approximately
every three months) to give users an explicit target for how long a
given release will be supported. Of course, there's an opportunity for
commercial support vendors to offer extended support :)

There would be some work required to adopt these ideas, but I think it
would result in a healthier development community that would be able to
focus on development and deliver enhancements to its users at a more
rapid pace.

Hopefully someone is still awake out there. I'd love to hear some
feedback.