Re: Release process, 2.15 and 2.16

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Tue, 21 Feb 2012, Roland McGrath wrote:

> > I don't think it makes sense to delay release tarballs indefinitely 
> > awaiting a resolution of test failures.  2.15 is tagged, it is what it is 
> > and the conclusion on those test failures cannot make any difference to 
> > what goes in the tarballs.  I think you should make and upload the 
> > tarballs now and go through the various steps for making announcements, 
> > updating websites etc. as described on 
> > <http://sourceware.org/glibc/wiki/Release>.
> 
> Conversely, I don't think it makes sense to declare and announce an
> official release when we know it to be buggy.  IMHO there's no loss
> if the next release is 2.15.1 and takes a few weeks more.

And conversely to that, trying to avoid *all* regressions is likely to be 
a recipe for endless delays and not getting releases out to users to whom 
they will be useful, and so not getting reports of problems that will come 
from those users.

We tried feature-based releases for GCC at one point, then moved to 
time-based releases - where we try to get the most important regressions 
fixed, but realistically don't expect to have them all fixed before the 
release.  We've been doing time-based releases for glibc for quite some 
time, and a consequence is that they should be good enough, not perfect.

>From my testing and from other testing done at Mentor on various 
architectures I'm happy that 2.15 release and branchpoint were reasonable 
points for a release and branching - good enough to be worth releasing, 
even if not perfect.  (2.4 was the only release I recall in the glibc 2.x 
(2.1 and later) era whose stability I'd have considered doubtful, and even 
that may somewhat be in retrospect not from real problems at the time.)

> > * As noted above, problems found after tagging should not delay the rest 
> > of the release process.
> 
> This is mostly an artifact of the previous procedure of tagging
> quasi-randomly before addressing stability, which we should change anyway.

Yes.

> Certainly, nothing needs to delay the non-committal steps of the process.
> But uploading a tarball and sending announcement email should not be done
> when the new release has known regressions from the last release.

I don't think known regressions are avoidable (and if you avoid them, you 
still have unknown regressions).

But we don't know what the failures Carlos is seeing are.  Do we even know 
if they are tests that were in 2.14, and if so whether they failed for 
2.14 in the same environment?  If they are new tests or failed for 2.14, 
they aren't regressions.

> > * The release manager should be the person doing the tagging so they get a 
> > release and branch in a state they find satisfactory rather than having to 
> > take it in whatever state it happened to be at a point determined 
> > independently of them.
> > 
> > * There should be an explicit stabilization period on master before 
> > tagging / branching, during which architecture maintainers are asked to 
> > ensure stability on their architectures and the release manager control 
> > what changes are considered appropriate to go in.
> 
> I tend to agree with these sentiments at the high level.
> But we need much more discussion and agreed concrete process
> to decide exactly how to work this.

I think we need to decide based on experience - given the high-level idea 
and whatever comments people make (e.g. those about keeping the 
stabilization period short, and what changes should be restricted then), I 
think the 2.16 release manager (e.g. Carlos if he does wish to do 2.16) 
should post plans and then we can see how people find them in practice and 
what we want to change for 2.17.  (In general I think we can give the 
release manager quite a lot of autonomy here.)

> This could be a reasonable start for the next cycle.  Obviously the next
> cycle is going to be the first instance of the new ways of doing things and
> thus a prototype for the cycles to come, and we'll hone the process in
> successive cycles.  But I'd like it to be done in the context of setting
> out the general formula for our schedules and procedures in the future.

My expectation is that an outline process on the wiki would be considered 
somewhat provisional - put up soon, revised as needed based on experience.

> Certainly this should be done.  The details of the preferred timing and so
> forth should be worked out with the Translation Project on the basis of our

I believe the TP advice is just to send releases and snapshots whenever 
ready, not to try a stronger coordination (it's up to each language team 
what they translate and when, anyway).

> approximate twice-a-year schedule, and of course thoroughly documented.

I put notes on the Release wiki page - of course, those are provisional 
until we actually have more experience and agreement, and will need 
expanding with more details of what to do.

> The practice in many projects is to have a "string freeze" that is similar
> to, but independent of other kinds of stabilization period.  This is the
> time when you update .pot files and submit them to the translators.  That
> is probably appropriate for us too, as is that it be earlier than the
> release stabilization proper, to allow translation teams time to work
> before the release.

That may make sense, though I think it would be enough just to submit the 
snapshot and not worry too much about whether there are string changes 
after then (we can always put new .po files on the release branch for 
translations updated after the .0 release).

-- 
Joseph S. Myers
joseph@codesourcery.com