Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor <cgf-use-the-mailinglist-please at cygwin dot com>]

On Mon, Mar 07, 2011 at 12:10:45PM -0500, Christopher Faylor wrote:
>On Mon, Mar 07, 2011 at 11:38:49AM -0500, Christopher Faylor wrote:
>>On Mon, Mar 07, 2011 at 10:37:08AM -0500, Christopher Faylor wrote:
>>>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>>>>On Mar  5 21:12, Robert Wruck wrote:
>>>>> Hi,
>>>>> 
>>>>> recently, I found that cygwin-git was not able to 'cat-file' files
>>>>> that exceeded some size (in my case about 80MB).
>>>>> I tracked this down to the cygwin implementation of write() that
>>>>> behaves quite odd in some cases.
>>>>> 
>>>>> I wrote a small program (source attached) that mmaps a given file
>>>>> and tries to write it to another file or stdout.
>>>>> 
>>>>> The results vary:
>>>>> 
>>>>> If the destination is a file (`writetest infile outfile` or
>>>>> `writetest infile > outfile`), the write succeeds in a single call.
>>>>> 
>>>>> If the destination is a pipe (`writetest infile | cat > outfile`),
>>>>> the write succeeds in most cases. BUT:
>>>>> 
>>>>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>>>>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>>>>> case 4096 byte for each call).
>>>>> This breaks git since it does an infinite loop while errno=EAGAIN.
>>>>
>>>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>>>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>>>>actually do that.  EAGAIN is only an expected error for non-blocking
>>>>I/O, so applications which use blocking I/O usually only test for EINTR.
>>>
>>>I can barely convince myself that there's a pathological case where an
>>>EAGAIN could leak out.  I'm investigating now.
>>
>>Actually, in this case, it looks like the problem is that Windows
>>doesn't like sending a huge buffer to a pipe.  The errno in this case
>>should probably be something like EFBIG rather than EAGAIN.
>>
>>Does git deal with this type of errno gracefully or does it just abort
>>if the write() fails for any reason?  I'd rather just fail and let the
>>caller deal with it than complicate Cygwin's code by trying to loop
>>writing smaller amount of data to the pipe so I'd prefer just changing
>>the errno if that solves the problem.
>
>Answering my own question: No, git doesn't make any useful decisions based
>on the errno.  So, just returning a different errno is not going to make
>this work.
>
>So, my options are to:
>
>1) Limit the buffer size to some value like 64 mib and expect the caller to
>deal with that.
>
>2) Limit the buffer size to some value like 64 mib and loop in
>fhandler_overlapped::write_overlapped until everything has been written.
>
>3) Attempt the WriteFile(), notice the error condition and then do 1) or 2).
>
>2) or 3) would be the most like Linux.  1) is by far the easiest and would
>probably solve the current git error case.
>
>Just thinking out loud here.  I think I've convinced myself that I should
>go with 3 + 2.

Except that it's tricky to get this right in the non-blocking case.  Sigh.

cgf