This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Question about madvise(DONTNEED) in glibc malloc


On 15 April 2013 06:39, KOSAKI Motohiro <kosaki.motohiro@gmail.com> wrote:
>> This doesn't prove that glibc use of MADV_DONTNEED is wrong.  What
>> this proves is that never giving memory back to the system results in
>> crazy fast performance since we reduce syscall overhead.  It doesn't
>> justify never returning memory back to the system though.
>
> It does. You need to look at current heap_trim() or you don't understand
> current DONTNEED design.
>
>>  extra = (top_size - pad - MINSIZE - 1) & ~(pagesz - 1);
>>  if(extra < (long)pagesz)
>>    return 0;
>>  /* Try to shrink. */
>>  if(shrink_heap(heap, extra) != 0)
>>    return 0;
>
> heap_trim() only check extra size is larger than page size.
>
> And Quote form man madvise.
>
>>       MADV_DONTNEED
>>              Do  not  expect  access  in the near future.  (For the time
>>              being, the application is finished with the given range, so
>>              the  kernel can free resources associated with it.)  Subse-
>>              quent accesses of pages in this  range  will  succeed,  but
>>              will  result  either  in  re-loading of the memory contents
>>              from the underlying mapped file (see mmap(2)) or zero-fill-
>>              on-demand pages for mappings without an underlying file.
>
> Current implemantation cleary does as document. It's not a bug and we don't plan to
> change never.

I didn't imply that it's any kind of bug in the kernel either.

Why can't we extend the TRIM_THRESHOLD to non-main arenas?  We
sacrifice the extra address space we could have given back to the
kernel by not getting rid of a heap when we could, but that's a much
better compromise IMO.

I agree with you when you say that the pages we just gave back with
MADV_DONTNEED are the ones most likely to be needed when we decide to
extend the heap next and in that context, it is a suboptimal use of
MADV_DONTNEED.  However, the idea here is to release resources
periodically to avoid monopolizing it.  Maybe we could always only
trim half of TRIM_THRESHOLD and keep the other half as a cache.  Given
the semantics of dynamic thresholds, we would effectively be keeping a
cache for the next expected allocation size - the TRIM_THRESHOLD is
dynamically set to twice of MMAP_THRESHOLD on freeing of a mapped
location.

Siddhesh
--
http://siddhesh.in


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]