Re: [RFA] bailing out of tests with multiple errors or timeouts

[Janis Johnson <janisjo at codesourcery dot com>]

On 05/04/2011 11:10 AM, Mark Kettenis wrote:
>> Date: Wed, 04 May 2011 10:46:50 -0700
>> From: Janis Johnson <janisjo@codesourcery.com>
> 
>> +    timeout {
>> +	# A timeout here often indicates more timeouts later in the test.
>> +	fail "$test (timeout)"
>> +	fail "Remainder of test"
>> +	return
>> +    }
> 
> Often, so not always.  Really the best way to get rid of these is to
> fix the underlying problem.

I agree, the best way to get rid of timeouts and ERRORs in gdb tests is
to fix gdb.  Unfortunately, that's not an easy task for a target whose
support is buggy or incomplete whether in gdb, simulator, hardware, or
operating system.  It's painful to test on such targets when a few tests
time out multiple times.

What I'm looking for is a way to bail out of tests that are known to
time out multiple times on some targets, or have a cascade of ERRORs.
That might involve changes to specific tests like the examples I posted;
it might involve setting a magic variable somewhere to bail out of any
test that times out or runs into an error; it might involve having a
list of tests that bail out after a timeout or error.  Ideally there
would be an easily-recognizable failure message for those tests to
indicate it's more than a normal failure; maybe UNRESOLVED.

I'm not familiar enough with the workings of the gdb test infrastructure
or the general conventions of the gdb testsuite to know what might be
appropriate, but presumably others have run into this problem and have
ideas.

Janis Johnson
CodeSourcery / Mentor Graphics