Re: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out (Re: GDB 8.2.90 available for testing)

[Pedro Alves <palves at redhat dot com>]

On 03/22/2019 02:42 PM, Simon Marchi wrote:
> On 2019-03-22 08:39, Pedro Alves wrote:
>> Hi Pedro,
>>
>> Thanks for the detailed report.
>>
>> On 03/07/2019 05:42 PM, Pedro Franco de Carvalho wrote:
>>
>>> I'm not sure if there is a proper way to work around this in GDB, but it
>>> would be useful so that the testsuite doesn't hang in these cases.
>>
>> I spent a while trying to fix this, and I came up with the patch below.
>>
>> WDYT?
>>
>> From 17e8f28072cce040524eab7b85b8767764a54cd2 Mon Sep 17 00:00:00 2001
>> From: Pedro Alves <palves@redhat.com>
>> Date: Fri, 22 Mar 2019 11:33:19 +0000
>> Subject: [PATCH] Fix testsuite hangs when gdb_test_multiple body errors out
>>
>> This commit fixes a regression in the testsuite itself, triggered by
>> errors being raised from within gdb_test_multiple, originally reported
>> by Pedro Franco de Carvalho's at
>> <https://sourceware.org/ml/gdb-patches/2019-03/msg00160.html>.  Parts
>> of the commit message are based on his report.
>>
>> This started happening due to a commit that was introduced recently,
>> and it can cause the testsuite to hang.
>>
>> The commit that triggers this is:
>>
>>  fe1a5cad302b5535030cdf62895e79512713d738
>>  [gdb/testsuite] Log wait status on process no longer exists error
>>
>> That commit introduces a new "eof" block in gdb_test_multiple.  That
>> is not incorrect itself, but dejagnu's remote_expect is picking that
>> block as the "default" action when an error is raised from within the
>> commands inside a call to gdb_test_multiple:
>>
>>   # remote_expect works basically the same as standard expect, but it
>>   # also takes care of getting the file descriptor from the specified
>>   # host and also calling the timeout/eof/default section if there is an
>>   # error on the expect call.
>>   #
>>   proc remote_expect { board timeout args } {
>>
>> I find that "feature" surprising, and I don't really know why it
>> exists, but this means that the eof section that remote_expect picks
>> as the error block can be executed even when there was no actual eof
>> and the GDB process is still running, so the wait introduced in the
>> commit that tries to get the exit status of GDB hangs forever, while
>> GDB itself waits for input.
>>
>> This only happens when there are internal testsuite errors (not
>> testcase failures).  This can be reproduced easily with a testcase
>> such as:
>>
>>   gdb_start
>>   gdb_test_multiple "show version" "show version" {
>>     -re ".*" {
>>        error "forced error"
>>     }
>>   }
>>
>> I think that working around this in GDB is useful so that the
>> testsuite doesn't hang in these cases.
>>
>> Adding an empty "default" block at the end of the expect body in
>> gdb_test_multiple doesn't work, because dejagnu gives preference to
>> "eof" blocks:
>>
>>         if { $x eq "eof" } {
>>         set save_next 1
>>         } elseif { $x eq "default" || $x eq "timeout" } {
>>         if { $error_sect eq "" } {
>>             set save_next 1
>>         }
>>         }
>>
>> And we do have "eof" blocks.  So we need to make sure that the last
>> "eof" block is safe to use as the default error block.  It's also
>> pedantically incorrect to print
>>
>>  "ERROR: Process no longer exists"
>>
>> which is what we'd get if the last eof block we have was selected
>> (more below on this).
>>
>> So this commit solves this by appending an "eof" with an empty
>> spawn_id list, so that it won't ever match.
>>
>> Now, why is the first "eof" block selected today as the error block,
>> instead of the last one?
>>
>> The reason is that remote_expect, while parsing the body to select the
>> default block to execute after an error, is affected by the comments
>> in the body (since they are also parsed).
>>
>> If this comment in gdb_test_multiple
>>
>>  # patterns below apply to any spawn id specified.
>>
>> is changed to
>>
>>  # The patterns below apply to any spawn id specified.
>>
>> then the second eof block is selected and there is no hang.
>>
>> Any comment at that same place with an even-numbered of tokens also
>> works.
>>
>> This is IMO a coincidence caused by how comments work in TCL.
>> Comments should only appear in places where a command can appear.  And
>> here, remote_expect is parsing a list of options, not commands, so
>> it's not unreasonable to not parse comments, similarly to how this:
>>
>>   set a_list {
>>      an_element
>>      # another_element
>>   }
>>
>> results in a list with three elements, not one element.
>>
>> The fact that comments with an even number of tokens work is just a
>> coincidence of how remote_expect's little state machine is
>> implemented.
>>
>> I thought we could solve this by stripping out comment lines in
>> gdb_expect, but I didn't find an easy way to do that.  Particularly, a
>> couple naive approaches I tried run into complications.  For example,
>> we have gdb_test calls with regular expressions that include sequences
>> like "\r\n#", and by the time we get to gdb_expect, the \r\n have
>> already been expanded to a real newline, so just splitting the whole
>> body at newline boundaries, looking for lines that start with #
>> results in incorrectly stripping out half of the gdb_text regexp.  I
>> think it's better (at least in this commit), to move the comments out
>> of the list, because it's much simpler and risk free.
> 
> Wow, this is top-notch TCL/expect/DejaGnu investigation skills.

Thanks.  Team work here, the initial analysis came from
Pedro Franco de Carvalho.

> I agree with just moving the comments out, since that's the correct thing to do.  Trying to strip them programmatically just adds some dark magic that can potentially introduce more weird behaviors.
> 
> But we need to remember this when writing tests, should we have an entry in the testcase wiki page?
> 
> https://sourceware.org/gdb/wiki/GDBTestcaseCookbook

That might be a good idea.

I'm merging the patch to master now.

Thanks,
Pedro Alves