Re: Question about madvise(DONTNEED) in glibc malloc

[Rich Felker <dalias at aerifal dot cx>]

On Sun, Apr 14, 2013 at 10:01:52PM -0700, KOSAKI Motohiro wrote:
> (4/14/13 9:28 PM), Rich Felker wrote:
> > On Sun, Apr 14, 2013 at 09:37:16AM -0700, KOSAKI Motohiro wrote:
> >> Hi all,
> >>
> >> Now, we linux MM folks discuss are discussing about new memory
> >> discarding feature. (https://lkml.org/lkml/2013/3/12/105). The
> >> motivation is similar wtih MADV_FREE, but more efficient.
> >> (http://lwn.net/Articles/230799)
> > 
> > If you're following this, do you know if vrange(VRANGE_NOVOLATILE) has
> > failure cases? 
> 
> If any pages have already been discarded, vrange(VRANGE_NOVOLATILE) return 1.
> Subsequent memory access may cause minor page fault and makes page zero fill.

But vrange never fails as long as the mapping is valid?

> > This makes a big difference to how usable it is by
> > malloc implementations. madvise re-mmapping new zero pages over the
> > range are nice in that the range is not using physical memory, but
> > still remains committed and immediately usable without any risk of
> > failure to get it back. 
> 
> Does you "commit" mean to talk about virtual address? If yes, vrange() never
> call neither mmap/munmap implicitly.
> If you are talk about physical pages, it may be discarded and disappeared. vrange()
> user must not pass undroppable data.

I mean in the sense of commit charge. I would assume vrange does not
adjust the commit charge, but this would be important to know.

> > If vrange can fail to "get back" the range,
> > this makes it a lot harder to use robustly.
> 
> Any linux syscall _can_ return ENOMEM if system has really no memory.
> but we've never seen practically because kernel handle memory starvation enough
> clever.

No, there are plenty of syscalls that never fail. And having this
property is important. For example, if calling vrange would have to
split up a range and thus add more ranges, the setting volatile mode
could reasonably fail, but setting non-volatile should just make the
entire contained range non-volatile so that no splitting is required
if it runs out of memory trying to do the split.

Rich