This is the mail archive of the gdb-patches@sources.redhat.com mailing list for the GDB project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFA:] sim-defs.exp: support xfail


Hans-Peter Nilsson wrote:
Date: Wed, 10 Nov 2004 09:01:56 -0500
From: Andrew Cagney <cagney@gnu.org>


Hans-Peter Nilsson wrote:

Directly taken from the same option in the ld run_dump_test.
Ok to commit?
2004-11-10  Hans-Peter Nilsson  <hp@axis.com>

* lib/sim-defs.exp (run_sim_test): Support "xfail" option.

What do you mean by "xfail"? (Hint, check a current dejagnu document where it describes kfail :-)


I know about kfail, but I'm agnostic.  I just mean what happens
for the ld run_dump_test "xfail" option; a simple setup_xfail.
The purpose is to be able to keep failing tests around but not
get the total results marked as partially failing.  Reasons for
the failure would be unspecified, matching both xfail and kfail
usage.

To me, it doesn't really matter whether it's kfail or xfail
(vivid descriptions of kfail vs. xfail to /dev/null).  In the
sim testsuite I believe it'll be used very sparingly, perhaps
not even for things that are checked in.  But it'd be Nice to
Have.

Is the change acceptable by naming the option "kfail" and
calling "setup_kfail"?  Or do you want both?

brgds, H-P
PS.  BTW, the dejagnu documentation on
<URL:http://www.gnu.org/software/dejagnu/manual/> is not
current; KFAIL isn't mentioned there.

Ok, really weird, the date on that page says 2004 but the manual is missing (from dejagnu 1.4.4):


@item KFAIL
A test is known to fail in some environment(s) due to a known bug
in the tool being tested (identified by a bug id string).  This
exists so that, after a bug is identified and properly registered
in a bug tracking database (Gnats, for instance), the count of
failures can be kept as zero.  Having zero as a baseline in all
platforms allow the tool developers to immediately detect regressions
caused by changes (which may affect some platforms and not others).
The connection with a bug tracking database allows for automatic
generation of the BUGS section of man pages or Release Notes, as
well as a "Bugs Fixed this Release" section (by comparing to a
previous release set of known failures).
The procedure @code{setup_kfail} is used to indicate a failure is
known to exist.

@cindex KFAIL, avoiding for POSIX
As with @code{XFAIL}, @sc{posix} tests must return @code{FAIL} rather
than @code{KFAIL} even if a failure was due to a known bug.

I can't think of a single reason for a simulator test that failing not being for a known bug.

However, for consistency with dejagnu, we might as well support both. Feel free to add the necessary glue to cover all of xfail and kfail and kpass.

Andrew


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]