This is the mail archive of the cygwin-developers@cygwin.com mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: The cygheap saga


At 12:18 PM 4/3/2004 -0500, Christopher Faylor wrote:
>On Fri, Apr 02, 2004 at 08:48:31PM -0500, Pierre A. Humblet wrote:
>>At 10:37 PM 4/1/2004 -0500, Christopher Faylor wrote:
>>>On Thu, Apr 01, 2004 at 09:36:14PM -0500, Pierre A. Humblet wrote:
>>>>At 06:11 PM 4/1/2004 +0200, Corinna Vinschen wrote:
>>>>>
>>>>>AFAICS, we have currently three pressing problems:
>>>>>
>>>>>- People complaining about cygheap problems (Win32 error 487) but
>>>>>  nobody willing to debug it.
>>>>
>>>>I will describe how it works (details in how-cygheap-works),
>>>>and then how we might change that to improve the chance of
>>>>success.
>>>>
>>>>1) When starting, Cygwin tries to reserve an amount of memory Pr
>>>>   as large as possible with CYGHEAPSIZE >= Pr >= CYGHEAPSIZE_MIN
>>>>   The initial committed memory size Pc is 0.
>>>>2) As Cygwin operates and calls csbrk , Pc grows. Pc can grow
>>>>   past Pr, although that is unlikely, and in fact impossible if
>>>>   Pr <  CYGHEAPSIZE. csbrk calls will not fail as long as Pc <= Pr.
>>>>3) When starting a child, the parent creates a file mapping of size 
>>>>   Max(Pc, Pr), copies Pc bytes to it and passes the handle to the child.
>>>>4) The child tries to map the region to the same address as the parent's
>>>>   heap. If that works, the child is done.
>>>>5) Else the child tries to reserve Max(Pc, Pr) at the same address
>>>>   as the parent's heap.
>>>>6) The child then commits and copies Pc bytes from the file mapping to
>>>>   that region.
>>>>The failure occurs in step 5.
>>>>
>>>>What I don't see if why we need to reserve Max(Pc, Pr) and not simply
>>>>a quantity Cr with Pr >= Cr >= Pc or Pr >= Cr >= max(Pc, CYGHEAPSIZE_MIN).
>>>>We could even try to make Cr as large as CYGHEAPSIZE, but if the parent
>>>>did not succeed I would think the child won't either.
>>>>As an aside, I am wondering if the current step 5 can ever work (on NT)
>>>>if step 4 has failed. Why would 4 fail if space is available? 
>>>
>>>Let me make it really simple.
>>>
>>>Parent is reserving X amount of space.
>>>
>>>Child can't reserve that much space because something (probably a DLL)
>>>is blocking the reservation of the full amount of space.
>>>
>>>Before we change the design, we need to find out what is blocking the
>>>reservation of the memory rather than just making the child allocate less
>>>space than the parent.
>>
>>Why not do both, given that this problem seems to pop up regularly?
>>At any rate I had written a patch and I tested on NT today (on a machine 
>>without problem). The curious can try it. It's a fairly simple patch.
>
>I wonder if cygheap should just be using techniques similar to fork and
>avoiding the use of shared memory.  The shared memory implementation was
>a noble experiment on my part.  IIRC, ReadProcessMemory actually turned
>out to be faster than using shared memory.

That may be faster but in absence of the "other thing" below, how does that
solve the problem of making sure that the heap address is  available in the
child?  Also wouldn't that force the long synchronization method where the
child must start suspended?

>The other thing I'd *really* like to do is somehow construct the DLL such
>that the N bytes of the cygheap are automatically reserved by the loader.
>I can't tell if this is impossible to do in PE format/Windows or if it's
>just not possible to do with GNU ld.

Yeah!

>If we could do that, it would probably eliminate many of the problems with
>rebase.
>
>If we need to minimize the amount of memory used in the child, we probably
>should generalize the routine currently used in init_cheap.

Do you mean decremending by 2 MB until it works? That's what I initially 
thought too (see my original message above), but when I started writing code
I changed my mind. One needs to pass more parameters to the child and I
doubt that calling VirtualAlloc many times until success is faster than
calling VirtualQuery once (assuming that idea works, which is still
unproven). 

>I do want to find out what's going on with people reporting this problem
>before I commit YA workaround, though.

OK. It looks like 1.5.10 is not for this weekend. I will send you a patch
for path.cc. I'd like to see it used for a few days before making it part
of an official release.

Pierre


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]