Re: [patch, testsuite] test newlib.string/strcmp-1.c takes too long

[Richard Earnshaw <rearnsha at arm dot com>]

On 02/05/12 09:19, Corinna Vinschen wrote:
> On May  1 17:49, Greta Yorsh wrote:
>> The test of strcmp in newlib.string testsuite can take too long to run on
>> simulators, resulting in timeout failures.
>>
>> This patch creates a short version of the test, by reducing the length and
>> the number of strings to compare. This patch also adds a new macro LONG_TEST
>> to control whether a short or a more comprehensive test should be used.
>> LONG_TEST is not set by default. 
> 
> I'm wondering if that's the right way to solve the problem.  In theory,
> whatever you set these values to, you could find a simulator which is
> hitting your timeout values.  Wouldn't it make more sense to raise the
> timeout values in the testsuite?
> 
> 
> Corinna
> 


But to what level?  The test is enormous (over 25 minutes of simulation
on one run we did), and probably overkill unless you're actively testing
a rewrite of strcmp.

I think defaulting to a cut-down version is a good idea.  It might still
be necessary to bump the timeout, but the overall cost during normal
testing doesn't need to be this high.  Don't forget that with multilib
testing you can end up running that test multiple times.

R.