Re: [GDB BuildBot] New "Try Server"

[Sergio Durigan Junior <sergiodj at redhat dot com>]

On Friday, July 29 2016, Antoine Tremblay wrote:

> Sergio Durigan Junior writes:
>
>> Last, but not least, your try build will generate its own testsuite
>> logs, which will be recorded in that builder's git repository, available
>> at:
>>
>>   <http://gdb-build.sergiodj.net/cgit>
>
> Humm won't that "pollute" the builder results ?
>
> I mean, if the builder is testing commit 1 2 3 and that those are
> commits that were done on master but there's a try patch coming in
> between named say 7 and it has higher priority.
>
> You will then have 1 2 7 3 being tested.
>
> Then when we want to check the results of 1 2 3 won't it be confusing to
> see 7 there ? Will there be an indication that it's a try patch ?

Yeah, that was my concern as well.  I took extra care when recording the
results on the git repo.

So, as you can see on the repositories for each builder I record the
following files:

  - baseline
  - gdb.log
  - gdb.sum
  - previous_gdb.sum

When a normal build is triggered, BuildBot will:

  - Copy the current gdb.sum to previous_gdb.sum

  - Perform the build

  - Upload the gdb.log file from the buildslave

  - Use the current gdb.sum to calculate the regressions against the new
    gdb.sum (generated by the testsuite)

  - Update the current gdb.sum with the contents of the new gdb.sum

  - Save everything on the repo

Now, when a try build is triggered, here's what will happen:

  - Perform the build

  - Upload the gdb.log file from the buildslave

  - Use the current gdb.sum to calculate the regressions against the new
    gdb.sum (generated by the testsuite)

  - Update the current trybuild_gdb.sum with the contents of the new gdb.sum

  - Save everything on the repo

So, as you can see, on a try build we don't mess with the files
necessary to calculate the regressions on a regular build.

> Also since the regressions are calculated from one build to the next
> won't that possibly be a problem if let's say build 7 introduces a FAIL,
> then build 3 has the same FAIL, but build 2 had a PASS ? We would then
> miss a regression on a master commit.

As explained above, we would still see the regression happening because
the gdb.sum file is not touched (just read) on a try build.

> Should we have separate try builders to avoid that?

My initial thought was that, but having more builders will pollute the
web interface (although there is probably a way to suppress them to be
displayed), and will only create more repositories on the buildmaster.
But if I had infinite resources, then yeah, more builders would probably
make sense :-).

> I'm also curious about what happens if you send it a series of patches,
> will it squash them ?

If you use the first method I explained (having a local branch and
invoking "buildbot try" without the "--diff" option), then it will
squash all your local commits into one patch.  If you use the second
method ("--diff" option), then IIRC you can only send one patch.

> In any case thanks for working on this :) I'm sure it will be quite
> useful.

My pleasure!

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/