Re: rsync over ssh hang issue understood

["Lev Bishop" <lev dot bishop at gmail dot com>]

On 6/28/06, Darryl Miles wrote:

> How do I pull down ssh/rsync/cygwin.dll and build in a way that I can
> see the problem ?

For ssh and rsync sources, use cygwin setup.exe. For cygwin.dll see
http://cygwin.com/contrib.html
Especially note the requirement to sign a copyright release, if you
want your patch to make it into the mainline cygwin.

> What debugging tools can I use ?

See how-to-debug-cygwin.txt
http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/how-to-debug-cygwin.txt?rev=1.12&content-type=text/x-cvsweb-markup&cvsroot=src

> Can I printf() the
> cygwin.dll ?

debug_printf() with strace

> I read further up the thread the issue maybe to do with
> winsup/cygwin/select.cc trying to provide emulation of select(2) to
> applications and that it doesn't know how to ask Win32 kernel if a
> NamedPipe is writable (without blocking). I presume the problem is the
> application code blocking when it did not expect it. Is it allowed for
> the cygwin.dll to create extra threads that are self managing ?
> Allowing for the IO to be offloaded to a worker thread, and while a
> thread of active for the emulated pipe fd then you revoke writability
> indications from select(2) interface. But if no worker thread is busy
> working on that fd then you get writability back ?

Yes, but it is very hard to get the precise unix semantics. For
example, the application issues a write() which spawns off a thread
that then blocks. Then the application exit()s, causing the thread to
also terminate before completing its write, and the write never
completes. You could get around this by using fork() instead of a
thread (very slow), or spawn() (somewhat more efficient) or
implementing all this in the cygserver (see winsup/cygserver/ ) which
will be much more efficient than fork()ing each write but will still
be slow due to the need to send everything first (via another pipe or
socket) to the cygserver and then to the pipe, and all the
context-switching this entails.

There is also the issue of what return value to give the application
doing the write() on the pipe. You'll have to be careful to deal with
error conditions, SIGPIPE, etc, etc.

> This allow part of CYGWIN.DLL to be offload the blocked WriteFileEx()
> call to a named pipe, so the function can return control to the application.

It's WriteFile() not WriteFileEx() AFAICT

> Would that be theoretically possible ?
>
>
>
> You dont need to create a new thread from every IO to a writable named
> pipe, you can keep a small number of threads around that will create
> themselves as necessary. Once a thread has completed the IO it can hang
> around for a short time, staying alive and going back onto an idle
> queue, blocking itself until more I/O was needed or a timeout occurs (5
> seconds). If a timeout occurs it removes itself from the idle list and
> destroys itself.  So you dont take the thread creation hit for every I/O.

There is already this type of capability. See cygthread.cc
Also take a look at fhandler_pipe::read() which does something
similar, to allow blocking read() on a pipe to be interrupted

Good luck,
Lev

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/