[Bug libc/5553] If mmap fails malloc may assert.

["drow at sources dot redhat dot com" <sourceware-bugzilla at sourceware dot org>]

------- Additional Comments From drow at sources dot redhat dot com  2008-01-08 17:00 -------
Created an attachment (id=2187)
 --> (http://sourceware.org/bugzilla/attachment.cgi?id=2187&action=view)
Proposed fix.

Sorry about the broken bug; I hit enter accidentally.

The failing assertion is this one:
  assert(!victim || chunk_is_mmapped(mem2chunk(victim)) ||
	 ar_ptr == arena_for_chunk(mem2chunk(victim)));

GDB's bigcore.c testcase triggers this assertion on several PowerPC systems I
tested.  It starts by a malloc too large for the system to satisfy; when
_int_malloc fails, malloc creates and tries a new arena.  This arena is saved
as the default arena for the main thread so future allocations come from that
arena instead of the main one.

Later the test tries a malloc which can be met by mmap.  Eventually mmap
returns ENOMEM after a number of similar allocations:

mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xa810c000
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb010d000
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 ENOMEM (Cannot alloca
te memory)
mmap(NULL, 134221824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 ENOMEM (Cannot alloca
te memory)
brk(0x180c1000) 			= 0x180c1000

I do not know why brk succeeded (another seven times, all 0x8000000 bytes) when
mmap failed.  But the result is a non-mmapped chunk allocated from the main
arena.	The assert checks the thread's specific arena and fails.  Updating
ar_ptr fixes the failure.

Patch attached.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=5553

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.