[Bug libc/167] malloc() eats excess ram

["bluefoxicy at linux dot net" <sourceware-bugzilla at sources dot redhat dot com>]

------- Additional Comments From bluefoxicy at linux dot net  2004-05-27 05:09 -------
Stuff about this can now be seen at

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=123965
http://bluefoxicy.blogspot.com/2004_05_01_bluefoxicy_archive.html#108507154064758877
http://sources.redhat.com/bugzilla/show_bug.cgi?id=167
http://sources.redhat.com/ml/libc-alpha/2004-05/msg00216.html

Here is a full pro/con outline from the current state of it.
Note that a "Segment" in context here is a span of related
used or unused memory (such that, if used, calling free() to
its beginning would free it); and a mapping is a
physical->virtual page mapping.

Pros:

 - In cases where an area of unused, allocated memory touches or crosses
   a page boundary, that area can be used for any allocation, including
   those larger than itself; those areas are of infinite size.
 - In cases where an area of used memory needs to be expanded, i.e. via
   realloc(), and crosses a page boundary, that area can be partially
   expanded in place, lessening the copying overhead.  The least
   overhead is attained by finding the highest boundary and inserting
   mappings ahead of it, after mmap()ing the original pages of the
   segment to be realloc()ed elsewhere.  Once the new mapping is made,
   all of the original pages containing only data for that segment can
   be munmap()ed.
 - In cases where an allocated page exists and is spanned entirely by
   an unused area of memory, that page can be freed to the system.
 - The internals of the allocator can track allocated and freed
   segments, in two separate lists.  The free segment list could be
   arranged by size; and all segments could have pointers to the
   entries to the next and previous segments.  Thus, tracking the
   beginning and the actual size would allow quick access to the
   entire list.
 - Mappings could be easily tracked so that any free() or any
   realloc() resulting in a move could check a physical page to
   determine if it is free, and then unmap all of the mappings to
   it if so.

Cons:

 - In the normal operation of a program, many physical pages will be
   mapped to two or more areas of virtual memory.
 - Resource limits could be hit on some systems, which would be
   handled by falling back to the heap
 - Some systems slow down after lots of mmap()ing.  Information I've
   gathered from kernel developers indicates that this would require
   several million physical->virtual mappings, at least, to be
   significant on Linux.

-- 


http://sources.redhat.com/bugzilla/show_bug.cgi?id=167

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.