Re: Recent simulator patches broke many sims

[Hans-Peter Nilsson <hans-peter dot nilsson at axis dot com>]

> From: Joel Sherrill <joel.sherrill@oarcorp.com>
> Date: Sun, 24 Mar 2013 15:45:27 +0100

> This came in after I was done email last night. My test
> run finished overnight with no horribly bad issues. I have no idea
> what the make check results should be though and they could be
> because I simply ran "make check" with no board specified and
> no gcc for the target installed.

This would be no news to *you*, but for the record:

You need a board (make check RUNTESTFLAGS=--target_board=$board
with e.g. board=cris-sim).  All boards are in "recent"
dejagnu-1.5 IIRC and most in ancient dejagnu-1.4.4.  You need
installed binutils (e.g. in some temp location added to PATH for
the duration of the test-run) for each sim configuration as
mentioned.  I don't run with target gcc; not needed for the
level of smoke test I'm after and I guess not for this change
either.

> There were some unexpected failures but I don't know what "truth"
> on that is.

The truth is in the baseline.

>  I have saved logs from the test run and will do so with
> the run on Mike's patch.
> 
> Starting another test run with Mike's patch.
> 
> I can put both sets of logs on an ftp server if someone wants to review
> them.  Guidance on what is a bad number of "unexpected failures" on
> a target is appreciated.

You compare against a baseline you create without your patches;
comparing old/new build logs should be sufficient.

Having said all that, I think just a successful *build* of each
sim would be sufficient here, if your grep can find no
sockser-specific tests in the target sim test-suites.

brgds, H-P