This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: Improving libm-test.inc structure and maintenance
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: "Joseph S. Myers" <joseph at codesourcery dot com>
- Cc: libc-alpha at sourceware dot org
- Date: Sun, 5 May 2013 15:04:14 +0200
- Subject: Re: Improving libm-test.inc structure and maintenance
- References: <Pine dot LNX dot 4 dot 64 dot 1305022244550 dot 12072 at digraph dot polyomino dot org dot uk>
On Thu, May 02, 2013 at 10:46:55PM +0000, Joseph S. Myers wrote:
> libm-test.inc is a 14000-line file, containing testcases for most libm
> functions where the inputs were generally manually chosen but the
> expected outputs were computed with MPFR or other tools. When adding
> libm tests, it is necessary to generate expected outputs and put them
> in the file; there is no standard automation for generating them. It
> is also necessary to generate the expectations for exceptions and
> errno setting, and similarly put them in the file, and likewise the
> conditionals for which tests are run for which floating-point formats.
>
> The tests are described by lines containing calls to function-like
> entities such as TEST_f_f. Those calls are interpreted by
> gen-libm-test.pl, which converts them to actual C code (libm-test.c)
> in the build directory.
>
> Most tests are run only in the default (round-to-nearest) rounding
> mode, although some are run in other rounding modes. If a given test
> is to be run in each rounding mode, it must be repeated four times in
> the file in separate functions, with the appropriate results for each
> rounding mode manually placed in each call to TEST_*.
>
Lack of rounding modes is probably because we did not respect them until
recently. Perhaps we can add flag to detect results that are correct
upto rounding.
> I propose two main changes to how these tests are handled:
>
snip
> I envisage two new checked-in files to support this, say
> auto-libm-test-in and auto-libm-test-out. The first file would
> contain a list of testcases, listing the function and the input
> arguments for each test. The second, generated by a new checked-in
> program (using MPFR and MPC), would also contain the expected
> results (including errno and exception settings) for each such test,
> for each floating-point format. The new program could either
> completely regenerate the file, or (the default) only generate
> results for newly added tests (full regeneration would take several
> minutes on a fast system - MPC is *slow* for some of the inverse
> trig / hyperbolic functions).
>
Why do auto-libm-test-out have to be checked in? If outputs be computed
automaticaly we can save them in test directory. I could write utility
that caches mpc results on disk and returns them.
> The test inputs might be listed as decimal or hex floats, or maybe
> as special values such as pi. In any case, each input would be
> converted to a set of floating-point values, by rounding up and down
> in each floating-point format. This would produce a set of tests
> (from which duplicates would be automatically removed). For each
> format, there would then be a subset of those tests for which the
> values are representable in that format, and those would be the
> tests run for that format. Other information that would be included
> with the inputs would be whether to disable some tests for
> TEST_INLINE; whether to disable for particular formats because of
> bugs; whether to allow spurious exceptions or permit expected
> exceptions to be missing because of bugs; similarly, whether to
> allow errno settings to be missing because of bugs.
>
> (There are six supported formats - flt-32, dbl-64, ldbl-96 (Intel),
> ldbl-96 (m68k), ldbl-128, ldbl-128ibm - though the differences
> between the two ldbl-96 variants only affect subnormals.
> ldbl-128ibm would be treated for the purposes of the generators as a
> format with a fixed 106-bit precision except for subnormals, except
> that LDBL_MAX is slightly smaller than it would be for an ordinary
> 106-bit format.)
>
> gen-libm-test.pl would then process auto-libm-test-out to generate C
> code inside the tables of tests for given functions, where they say
> (for example) AUTO_TESTS (cos). So the tables would combine manual
> tests (for NaNs, infinities, etc.) and automatic ones (for finite
> inputs and outputs - though the finite outputs may overflow /
> underflow for particular formats). Results would automatically be
> generated for all rounding modes; the C code in libm-test.inc would
> at least initially be responsible for running tests in each mode
> where desired (appropriate macros / functions might be added to make
> it easy to do so).
>
> I suggest that the "names" of the tests that go in libm-test-ulps just
> name the function, rounding mode (as applicable) and inputs (as hex
> floats, removing the existing support for prettier names of certain
> test inputs), but not the expected output. This would of course
> invalidate existing ulps and require them to be regenerated from
> scratch, but apart from x86/x86_64 we try to do such regeneration
> before each release anyway, and it's probably a good idea for
> x86/x86_64 despite the machine-dependence.
>
> I'd envisage changes such as the above being implemented incrementally
> - first setting up support for tables of tests and moving tests into
> such tables bit by bit, before then setting up support for automatic
> test generation and moving existing tests into it. This means that
> initial versions of support for tables, or for automatic generation,
> don't need to support all features (disabling tests for particular
> formats, missing exceptions, ...); features can be added when needed
> to convert tests for a particular function.
>
> It is of course possible also to split up libm-test so that the tests
> for each function are run separately (with or without also splitting
> up the auto-libm-test-* files). I don't know if that's desirable.
>
One possibility would be use dryrun format which also could allow
automate bug report of inaccurate results. I could combine this with
program that tries random values and looks for inaccuracies.
> --
> Joseph S. Myers
> joseph@codesourcery.com