Re: Crashes in x86_64 math functions

[Allan McRae <allan at archlinux dot org>]

On 09/01/12 23:43, Arkadiusz MiÅkiewicz wrote:
> On Monday 02 of January 2012, Allan McRae wrote:
>> I am seeing some crashes in subversion and apache from various math
>> functions in glibc-2.15 on x86_64 with backtraces like:
>>
>>
>> #0 0x0000000000005446 in ?? ()
>> #1 0x00007fffebddb095 in floor () from /lib/libm.so.6
>> #2 0x00007ffff7de805c in _dl_relocate_object () from
>> /lib/ld-linux-x86-64.so.2
>> #3 0x00007ffff7dee526 in dl_open_worker () from /lib/ld-linux-x86-64.so.2
>> #4 0x00007ffff7dea286 in _dl_catch_error () from
/lib/ld-linux-x86-64.so.2
>> #5 0x00007ffff7dede6a in _dl_open () from /lib/ld-linux-x86-64.so.2
>> #6 0x00007ffff66fef26 in ?? () from /lib/libdl.so.2
>> #7 0x00007ffff7dea286 in _dl_catch_error () from
/lib/ld-linux-x86-64.so.2
>> #8 0x00007ffff66ff4cf in ?? () from /lib/libdl.so.2
>> #9 0x00007ffff66fefc1 in dlopen () from /lib/libdl.so.2
>> #10 0x00007ffff7742e5b in apr_dso_load () from /usr/lib/libapr-1.so.0
>> ...
>>
>> I bisected the issues to commits:
>>
>> Use rounds{s,d} for x86 rint, ceil, floor
>> http://sourceware.org/git/?p=glibc.git;a=commit;h=ad0f5cad
>> (crashes in floor/ceil)
>>
>> Optimize accurate 64-bit routines for FMA4 on x86-64
>> http://sourceware.org/git/?p=glibc.git;a=commit;h=af968f62
>> (crash in sin/tan)
>>
>>
>> Not all x86_64 machines are affected so I figured it was probably
>> multiarch stuff and this seems confirmed by the issues being "fixed" by
>> deleting the relevant files in sysdeps/x86_64/fpu/multiarch.
>>
>> This is using glibc-2.15 built with gcc-4.6.2 (20111223), binutils
>> 2.22.0 (20111227) and linux-3.1.5.
>>
>> Is anyone else seeing similar issues?
>
> I'm seeing same crases and archlinux patch helps for these.

I really can not recommend that patch given it was made with very little
understanding of the real issue.

> Was there anything new found by debugging in meantime?

Not really.  I am not even really sure this is an issue with the math
routines given they work fine when tested on their own.  Also, I'm not
sure why _dl_relocate_object would need floor/ceil/sin/tan.  Looking at
valgrind output seems to confirm this somewhat:

==19192== Jump to the invalid address stated on the next line
==19192==    at 0x5446: ???
==19192==    by 0x11A3E5: dl_open_worker (in /lib64/ld-2.15.so)
==19192==    by 0x116145: _dl_catch_error (in /lib64/ld-2.15.so)
==19192==    by 0x119D29: _dl_open (in /lib64/ld-2.15.so)
...

I have found the same backtrace for two projects that are using Fedora
16 (which essentially has glibc-2.15):

http://sourceforge.net/tracker/index.php?func=detail&aid=3468312&group_id=96039&atid=613414

http://comments.gmane.org/gmane.comp.emulators.libvirt/49492

Note the fix in that second link involves changing RTLD_NOW to RTLD_LAZY
in the dlopen() call, which further indicates to me that it is not
really an issue with the math functions.

I have not been able to debug this further yet due to a combined lack of
time and ability...

Allan