Re: emacs and large-address awareness under recent snapshots

[Ryan Johnson <ryan dot johnson at cs dot utoronto dot ca>]

On 09/08/2011 11:21 AM, Corinna Vinschen wrote:
> On Aug  9 10:23, Ken Brown wrote:
> > On 8/9/2011 10:12 AM, Ken Brown wrote:
> > > On 8/9/2011 7:19 AM, Ken Brown wrote:
> > > > (gdb) thread 1
> > > > [Switching to thread 1 (Thread 19828.0x447c)]
> > > > #0  0x00622ee0 in morecore_nolock (size=1052672) at gmalloc.c:703
> > > > 703           while ((__malloc_size_t) BLOCK ((char *) result + size)>
> > > > newsize);
> > > > (gdb) p /x size
> > > > $1 = 0x101000
> > > > (gdb) p /x heapsize
> > > > $2 = 0x80000
> > > > (gdb) p result
> > > > $3 = (void *) 0x807d0000
> > > > (gdb) p newsize
> > > > $4 = 0
> > > > (gdb) p _heapbase
> > > > $5 = 0x816000 "\202"
> > > > (gdb) p _heapinfo
> > > > $6 = (malloc_info *) 0x80060000
> > > >
> > > > Is _heapbase the problem?  This is initialized to _heapinfo at the first
> > > > call of malloc and is never changed.  _heapinfo presumably points into
> > > > the static heap at that point.  (_heapinfo is later changed as a result
> > > > of realloc.)  This low value of _heapbase is used in the BLOCK macro.
> > > Here's what I think is happening.  When temacs.exe is running during the
> > > build process (see my explanation of this earlier in the thread),
> > > malloc_init is called and _heapbase is set.  At this point, temacs is
> > > using its own static buffer as the heap, and _heapbase gets the value
> > > 0x816000.  This gets dumped as initialized data into emacs.exe, as does
> > > the value __malloc_initialized = 1.  Now when emacs.exe is run, it sees
> > > that malloc has already been initialized, so _heapbase retains its
> > > value, which is no longer appropriate.  All code relying on the BLOCK
> > > macro is now invalid.
> > >
> > > AFAICS, this has always been wrong.  But the error didn't have dramatic
> > > consequences until the heap was put into high memory.
> > >
> > > I'm not sure what's the best way to fix this (assuming my analysis is
> > > right).  Would it be enough to set __malloc_initialized to 0 before
> > > dumping?  That would force emacs to reinitialize and get the correct
> > > value of _heapbase.
> > No, that's too simple-minded.  I just tried it, and emacs aborted.
> > This seems like a mess.
> What happens if you remove the Cygwin-specific call to bss_sbrk in
> __default_morecore?  In theory that should also break, as long as
> temacs isn't also build large address aware.  The only difference,
> _heapbase = 0x20000000.  But if temacs gets build with large address
> awareness set, _heapbase should become 0x80000000.
>
> However, whatever you do, it will not really work.  Keep in mind that
> the large address awareness only makes sense (and has any effect!) on
> systems which provide a large address area.
>
> To me the bottom line here is, that emacs is doing the wrong thing.
> There are a couple of assumptions how a system maintains memory, which
> are just not valid on all systems.  The malloc initialization and the
> assignment of the heapbase (the first call to sbrk(0)) should happen
> in emacs every time it starts.
I'm pretty sure emacs [thinks it] doesn't even use the system heaps 
(sort of how cygwin doesn't use the windows heaps); from what I 
remember, the "heap" in [t]emacs is an .idata section of the image (12MB 
large on my version of emacs) which is supposed to have unused address 
space afterward, similar to how cygwin allocates its heap. There's even 
a comment there that says they got the idea from cygwin.

Does anybody know why emacs is accessing anything at 0x80000000 in the 
first place?

Ryan


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple