[OPEN-ILS-DEV] Cutting releases

Thu Jan 17 11:00:00 EST 2013

On Wed, Jan 16, 2013 at 11:43 PM, Dan Scott <dan at coffeecode.net> wrote:

> So, on the bright side, we got three releases out today, and turned
> around a fix for a critical security problem in minimal time in a
> collaborative fashion, and had blog posts and everything.
>
> I have a few concerns, though, which at the risk of sounding overly
> negative I'll list out here...
>
> I saw no mention of testing the release tarballs on various distros
> before they were released. I had asked for sanity testing of my 2.1.5
> tarball but didn't get any response. Security releases suck but still
> need QA. How much sanity testing did these most recent releases get?
>

You ask the toughest question first, so please stay with me after this
answer.

I'm not going to avoid looking this question in the eye.  I generally
confirm a successful build, start the staff client, and test the database
upgrade script (all on one distro only, plus Windows for the staff client).
 I don't believe any significant number of Evergreen releases, in my hands
or anyone else's, get any more direct testing than that.  I know that many
releases have had less.

If code gets tested when merged master, I believe we've rested on that as
an excuse to act as if releases are just a detail of publishing code that's
already likely to work.  I am aware of how lame that is (really), but I
also don't know where we're going to get more QA without making users wait
unreasonably long times for releases.

Despite major, laudable efforts from diverse contributors, who do we have
who can drop everything to put a package (or several) through its paces
every month?  We have to improve, I agree, but it's not clear to me how we
meet timed release goals, which we all agree are good, and at the same time
wait for proper testing.  Where is the manpower?

In the spirit of being constructive, maybe we could move back release
cutting by a week, but keep the date when we actually release them the
same?  More testing would be possible then.  Is this the kind of thing you
might go for?

I thought the outcome of one of the last meetings was that we were going
> to adopt a more open security process, one that would enable community
> members to contribute towards testing?
>
>
Security releases seem to invoke our lizard brains, and we just act to get
these releases out fast.  We need to change that, and if we concretized
those ideas from the recent meeting, let's bring that document out front
and center to remind ourselves.

Perhaps due to the lack of QA, our 2.2 and 2.3 tarballs once again
> contain the full xulrunner binary packages for both win32 and
> linux-i686, bloating their sizes by tens of megabytes. Bloat aside,
> for legal reasons it's generally not advisable to ship binaries
> from other projects if you can avoid it.
>

By "once again" I can tell there has been a conversation about this before,
which I must have missed, but I was not aware of this problem at all.  I
agree, let's avoid packaging these binaries going forward.

>
> I think our release processes need some work.
>
> Exhibit A: Even for a security release, we should be able to prep a
> git branch over in the security repo, cut a tarball from it, and
> then when the release is announced, just "git push origin
> security/rel_2_1_5:rel_2_1"
> so that the upstream release branch is a perfect replica of what went
> into the tarball.
>
Instead, at least for the rel_2_1 branch, two times today
> commits went into the rel_2_1 branch despite my having painstakingly
> prepped a complete branch over in the security repo last night (which
> also included commits from rel_2_1_4 that were missing from rel_2_1). I
> found that pretty frustrating, although I certainly overreacted today
> (probably due to staying up until 2:00 AM to prep the materials for the
> release and not getting enough sleep to act like a decent human being).
>

Neither Bill nor I were aware of the importance of the perfect replica.  We
weren't trying to wreck your work, we just didn't understand that was a
thing.  We probably need further discussions, in IRC perhaps, to make sure
we understand the most correct way of structuring the tags in relation to
the release branch.  I'm also unclear on how we deal with the generated
ChangeLog that goes into releases, but presumably shouldn't go into the
rel_2_n branch.

>
> Exhibit B: We seem to keep stumbling over how to keep upgrade scripts in
> sync. Back when I proposed the version-upgrade directory as a means
> of stamping out the problems we had historically had with fixes for
> point-to-point scripts not making their way into all the right
> releases, I also proposed that all point-to-point version upgrade
> scripts should be committed first to master, then backported to each
> applicable release. Can we commit to taking this approach? Or is there a
> better alternative?
>

We can, and should.  We need to change the make_release script, which
creates the upgrade script and commits it in such a way as to encourage us
to push it to the wrong place first.  Is this also possibly the right
answer for the generated ChangeLogs?

> Exhibit C: Open-ILS/src/perlmods/lib/OpenILS.pm keeps saying 'our
> $VERSION = "2.00";' - in rel_2_1 land I've been trying to remember
> to rev that (although I think the 2.1.5 release actually says 2.14 -
> sigh).  The release script is all well and good--I believe in better
> living through automation--but the script needs regular injections of
> human intelligence too. See also QA above! (Aside--what happens when we
> get to 2.2.10? Maybe we should be using 2.015, 2.025, 2.033 instead of
> 2.15, 2.25, 2.33...)
>
> So... maybe we can document our release process a bit more, with a
> checklist of steps to follow to create a tarball and of things to QA?
>

Yes.  We need a human script first.

>
> Exhibit D: General confusion around translation processes. They're a
> black box for most people, they don't result in quality output for
> groups serious about their translations, there's an impedance mismatch
> between the tools some translation groups want to use and between casual
> translators... we could really use a dedicated translation coordinator
> to step up and sort out some improvements on this front. Any volunteers?
>

I'm not the volunteer, but I second your call for one.  I would like to
have further talks about the translation merging processes.  Today I follow
them only mechanically, with little understanding of how they work.  I'd
like it if all the release managers/maintainers could talk to you about the
big picture of how they work.  I think you're the only one here who at
least mostly gets it.

> On tagging releases:
>
> I've brought this up in IRC a few times, but as far as I can tell the
> only reason that we're not using actual "git tag" to create annotated
> GPG-signed tags for releases is because one of the git admins doesn't
> like git tags. I have to say thanks to Galen for having documented
> the procedure for OpenSRF, which has worked well for me over there to
> date, and I wish we would use actual tags in Evergreen instead of
> branches that are named "tag/blah".
>

I assume the resistance is based on the idea that we're likely to cut a
release and then need to make a quick correction when something has been
missed (if not that, then what?)  I think others of us (me anyway) are just
neutral on this question, because we aren't aware of any hard advantages of
real, signed tags other than a vague idea of "correctness."  I am willing
to be enlightened.

-- 
Lebbeous Fogle-Weekley
 | Software Developer
 | Equinox Software, Inc. / The Open Source Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  lebbeous at esilibrary.com
 | web:  http://www.esilibrary.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20130117/4288e7ee/attachment-0001.htm>