This is the mail archive of the glibc-bugs@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[Bug libc/4943] Inconsistent rounding behaviour for sprintf and IEEE doubles


------- Additional Comments From redhat dot com+Bugzilla at edp dot org  2007-10-05 19:20 -------
Paul Sephton's statements are consistent with a 15-decimal-digit model of
arithmetic and a non-standard rounding rule.  E.g., suppose sprintf behaved
this way when passed a floating-point format string and an IEEE 754
double-precision number x:

	Set y to the 15-decimal-digit number nearest to x.  (This requires y be
	maintained in some format capable of representing decimal numbers.)  (To
	simplify discussion, I omit consideration of underflow and overflow.)

	Round y according to the format string, with ties rounded away from zero.

I believe this would produce the behavior he desires.

Of course, neither of these is the way that IEEE 754 floating-point arithmetic
or C's sprintf function is specified to behave.

IEEE 754 defines a floating-point number in terms of a sign s, a biased
exponent e, and a fraction f.  It also refers to a significand, which, for
normal numbers, equals 1+f.  Paul Sephton used the term "mantissa," but that is
incorrect.  A mantissa is the fractional part of a logarithm.  If x is a
normally representable positive number, E is an integer, and 2**E <= x <
2**(E+1), then the significand of x is x / 2**E and the mantissa of x is
log[b](x) - floor(log[b](x)), for some logarithm base b, often ten.

Quoting ANSI/IEEE Std 754-1985, section 3.2.2, with some changes due to limited
typography, the value of a double-precision number is "(-1)**s * 2**(e-1023) *
(1.f)".  That is the exact value represented.  That is the entirety of the
representation.  IEEE 754 says nothing about considering it to be a
15-decimal-digit number.  Any assertion that an IEEE 754 floating-point number
represents other numbers, such as a small interval around the number, has no
basis in the standard.

I will use the hexadecimal floating constant notation defined in C ISO/IEC
9899:TC2 6.4.4.2.  In this notation, a number 0x1.234p56 stands for 0x1.234 *
2**56, that is, (1 + 2/16 + 3/16**2 + 4/16**3) * 2**56.

Nothing in the 1985 IEEE 754 specification indicates that double-precision
numbers stand for 15-decimal-digit numbers.  According to IEEE 754, if a
double-precision number has sign bit 0, exponent 11, and fraction:

	0x.44b0ccccccccd,
	0x.44b4, or
	0x.44b7333333333,

then the floating-point number represents, respectively:

	1 * 2048 * 0x1.44b0ccccccccd,
	1 * 2048 * 0x1.44b4, or
	1 * 2048 * 0x1.44b7333333333.

In decimal, the number is exactly:

	2597.52500000000009094947017729282379150390625,
	2597.625, or
	2597.72499999999990905052982270717620849609375.

Observe that this completely explains the existing sprintf behavior:

	"2597.525" in source is translated at compile time to the floating-point
	number 0x1.44b0ccccccccdp+11.  This number is passed to sprintf to be
	converted according to the format string "%.2f".  sprintf examines
	0x1.44b0ccccccccdp+11, which is
	2597.52500000000009094947017729282379150390625, and it has a choice of
	rounding it to "2597.52" or "2597.53".  Since
	2597.52500000000009094947017729282379150390625 is closer to 2597.53,
	sprintf rounds to "2597.53".

	"2597.625" in source is translated to 0x1.44b4p11, which is 2597.625.
	Given the choices of "2597.62" and "2597.63", they are equally far from
	2597.625.  sprintf uses its rule for ties which is to round to even, so
	it produces "2597.62".

	"2597.725" in source is translated to 0x1.44b7333333333p11, which is
	2597.72499999999990905052982270717620849609375.  Given the choices of
	"2597.72" and "2597.73", 2597.72 is closer, so sprintf produces "2597.72".

This also demonstrates there are two problems producing the behavior Paul
Sephton desires.  One is that sprintf rounds ties in a way Paul Sephton does
not like.  The second problem is that IEEE 754 double-precision does not
represent 15-decimal-digit numbers exactly.  We can see this because even if
sprintf's rule for ties were changed, "2597.725" in source results in passing a
number smaller than 2597.725 to sprintf, so there is no tie, and sprintf rounds
it down.

Is there any basis for adopting a 15-decimal-digit model of arithmetic?  This
is not part of the 1985 IEEE 754 specification.  Paul Sephton does not cite any
source for his statements regarding the behavior of floating-point arithmetic.
The IEEE 754 standard says that floating-point numbers represent (-1)**s *
2**(e-1023) * (1.f).  It does not say they represent anything else.  Nothing in
the standard tells us that 0x1.44b0ccccccccdp+11 represents 2597.525.

There is no basis for treating IEEE 754 floating-point numbers as
15-decimal-digit numbers.

It is true that many people think of 2597.525 as being represented by
0x1.44b0ccccccccdp+11.  When they type "2597.525" into source or in a text
string that is converted to a double-precision number, they get
0x1.44b0ccccccccdp+11.  I expect this result leads them to think that
0x1.44b0ccccccccdp+11 represents 2597.525.  Nevertheless, it does not.
0x1.44b0ccccccccdp+11 is only an approximation of 2597.525.  It is actually
itself and is not the thing it approximates for a particular user.

When sprintf receives 0x1.44b0ccccccccdp+11 or 0x1.44b7333333333p11, it has no
way of knowing the user intends for these to be 2597.525 or 2597.725.  It can
only interpret them with the values that IEEE 754 says they have, which are
2597.52500000000009094947017729282379150390625 and
2597.72499999999990905052982270717620849609375.

Paul Sephton stated, "I would assume that Microsoft does at least loosely
follow a standard."  "Assume" is the correct word; he does not give any basis
for this belief.  I know that Microsoft C neither conforms to the 1999 C
standard nor attempts to, and I do not have information that it conforms to the
earlier C standard or to IEEE 754.

Paul Sephton wrote: "Please.  I really really really want a solution here."
>From the above, we know what the two problems are and we can offer a solution:
sprintf is obeying its specifications and will not do what you want.  You must
write your own code to produce the behavior you desire.

I suspect the behavior Paul Sephton desires can be produced in his application
by passing x * (1 + 0x1p-52) to sprintf in lieu of x.  This makes certain
assumptions about the nature of the values he has -- it is not the same as
converting the double-precision argument to 15 decimal digits and then
rounding, but it may produce the same results in his situation.  If a
double-precision number passed to sprintf is the double-precision number
nearest a 15-decimal-digit number, then I think (have not checked thoroughly)
that passing x * (1 + 0x1p-52) instead will yield the same result as if sprintf
were somehow passed the 15-decimal-digit number in a decimal format.  This is
because the replacement number corrects both problems:  It changes 2597.625 to
a slightly higher number, avoiding the problem with ties, and it changes the
double-precision number just below 2597.725 to a number slightly higher,
avoiding the second problem.  However, if a number being passed to sprintf is
more than one ULP away from a 15-decimal-digit number, this method can fail.

Paul Sephton refers to "the rules of decimal mathematics" but gives no
citation for the rounding rule he proposes.

In summary:

	If we use IEEE 754's statements about the values that double-precision
	numbers represent, then sprintf's behavior is fully explained.

	If we use Paul Sephton's statements about floating-point numbers being
	15-decimal-digit numbers, then they conflict with IEEE 754 and are
	inconsistent with sprintf's behavior.

This speaks for itself and is entirely sufficient to resolve the matter.

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=4943

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]