Re: Mauve wishlist

[Tom Tromey <tromey at redhat dot com>]

>>>>> "David" == David Gilbert <david.gilbert@object-refinery.com> writes:

>> Another suggestion that Tom Fitzsimmons had was to change the way we
>> count the number of tests.  Counting each invocation of the test()
>> method rather than each call to harness.check() has two benefits:

David> We can work around that by ensuring that all the tests run linearly
David> (no if-else branches - I've written a large number of tests this way
David> and not found it to be a limitation, but I don't know what lurks in
David> the depths of the older Mauve tests). There is still the problem that
David> an exception being thrown during a test means some checks don't get
David> run, but a new Mauve comparison report (not yet developed, although
David> I've done a little experimenting with it) could highlight those.

I've always tried to write tests the way you suggest, but the
exception problem turns out to be a real one, preventing test
stability in some cases.

One thing I like about this current proposal is that it automates test
stability -- the only failure modes possible are if a test hangs or if
the VM crashes.

As far as having more granular information -- we can still print a
message when a check() fails.  A command line option to the test
harness could control this, for instance.  I think we don't want to
just print a plain 'FAIL', we want some explanation; the detailed info
could go there.

Tom