Re: Cygwin TCP slow

[Brian Inglis <Brian dot Inglis at SystematicSw dot ab dot ca>]

On 2016-12-02 13:29, Daniel Havey wrote:
> On Wed, Nov 30, 2016 at 1:13 PM, Lee <ler762@gmail.com> wrote:
>> I don't know if this qualifies as a simple test case, but if you
>> don't already have wireshark, get it from
>> https://www.wireshark.org/download.html
>> get iperf-2.0.9.tar.gz from https://sourceforge.net/projects/iperf2/files/
>> change the setsockopt calls on lines 125 & 132 of
>> src/tcp_window_size.c to
>>   rc = 0;
>> ./configure  --prefix=/usr/local
>> make && make install
>> start wireshark & add a column for bytes in flight:
>> edit / preferences
>> appearance / columns
>> click on the "+" button to get a new row
>> double click on the "New Column", type BIF
>> double click the empty bit in the Fields column
>> type tcp.an which pulls up a pick list; click on
>> tcp.analysis.bytes_in_flight
>> click on OK
>> find a public iperf server -- I got lucky & found one ~65ms away,
>> so thruput is going to be constrained by the 130ms round trip
>> time. start the wireshark capture
>> iperf  -c <host name>
>> when it's done stop the capture & click on the BIF column header
>> What I get is a max bytes in flight of 66560
>> ----recompile iperf using the windows cross-compiler
>> make clean
>> make distclean
>> ./configure  --prefix=/usr/local --build=i686-pc-cygwin --host=i686-w64-mingw32
>> make && make install
>> start capturing
>> iperf  -c <host name>
>> when it's done stop the capture & sort on the BIF column
>> What I get is a max bytes in flight of 196608
>> So, for me, it's about a 3X difference between the native & cygwin
>> app.
>> If the problem really is in net.cc as the OP said, have a look at
>> https://cygwin.com/git/gitweb.cgi?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/net.cc;h=e4805d3e11c3cea09b1cdfa27170dfe626265125;hb=HEAD
>> starting at line 587
>> /* Raise default buffer sizes (instead of WinSock default 8K).
>> I think starting with Windows Vista the default is tcp autotuning.
>> So unless there's some other reason for setting the send/receive
>> buffer it seems that cygwin apps would be better off going with the
>> defaults.

> 0.) I don't care about XP at all and I care only a little about
> other downlevel products. (The further downlevel the less I care :)).
> It's Windows 10 from now until forever. There will be no eleven.
> 1.) Yes, I am the program manager for Windows TCP/IP and I want to 
> make the Internet a better place for everyone by setting the
> transport buffers correctly.
> 2.) Here is the cygcheck.out file with all that information. The make
> is failing because of some device configuration.
> 
> /home/dahavey/oss/src/newlib-cygwin/winsup/cygwin/gendevices: shilka
> command missing? - No such file or directory
> make[3]: *** [Makefile:720:
> /home/dahavey/oss/src/newlib-cygwin/winsup/cygwin/devices.cc] Error 2
> make[3]: Leaving directory
> '/home/dahavey/oss/build/x86_64-unknown-cygwin/winsup/cygwin'
> make[2]: *** [Makefile:81: cygwin] Error 1
> make[2]: Leaving directory
> '/home/dahavey/oss/build/x86_64-unknown-cygwin/winsup'
> make[1]: *** [Makefile:9464: all-target-winsup] Error 2
> make[1]: Leaving directory '/home/dahavey/oss/build'
> make: *** [Makefile:883: all] Error 2
> 
> /home/dahavey/oss/src/newlib-cygwin/winsup/cygwin/gendevices doesn't 
> exist in the sources that I downloaded from the cygwin website
> using: git clone git://sourceware.org/git/newlib-cygwin.git
> Something is not right here :).
> 
> 3.) No point in going further until this problem is sorted out.

Install cocom package which includes shilka, and other utilities named 
after Soviet weapons. See newlib-cygwin/winsup/doc/fhandler-tut.txt
 
> 4.) This is a little random to the thread, but, does anybody know
> who I might talk to about getting email addresses taken out of the
> spam filter on the mailing lists? I suspect that every address 
> @microsoft.com is being filtered. I don't know why this happened, 
> but, it does seem a little draconian to spam filter all 100,000 of
> us :). Also it's a pain in the but to use @gmail for business and it 
> further slows the process.

See https://sourceware.org/lists.html for policies and contacts. 
Only plain text *ONLY* email is accepted - HTML, some MIME content, 
some attachments, confidentiality, privacy, or disclaimer notices will 
get email bounced or just undelivered, as they don't want confidential, 
private, risky, or unreadable content posted, and can't guarantee it 
will get to the intended recipient if list members unsubscribe. ;^>
This often disqualifies using corporate email accounts for 
lists/groups with similar (auto-)moderation policies.
See http://www.frontierfleet.net/email/ for rationale, blocks, and 
workarounds. 

> 5.) Just FYI: I want the buffering done correctly for all apps that 
> use Windows so I am considering changing the behavior of Windows (no 
> downlevel support) to not mess up the buffers even if the app sets 
> SO_RCVBUF and/or SO_SNDBUF. This method is cool because it fixes the 
> problem for everyone who uses the new Windows builds, but, I also 
> don't like to change standard behavior.

Do Windows versions and flavours have accessible feature codes 
indicating when RFC 1323 TCP Window Scaling is available, active, and 
its parameters? I could see custom builds modifying desktop or VM 
setups to limit damage and load e.g. Education, Enterprise, and 
Insider messing around; and companies, vendors, and users setting GPOs 
or reg entries for apps that perform worse with Window Scaling and 
RFC 1323 non-compliant (old) equipment compatibility e.g. some 
Education and non-profit orgs.

$ ls /proc/registry/HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Internet\ Settings/WinHttp/TcpAutotuning
ls: cannot access '/proc/registry/HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Internet Settings/WinHttp/TcpAutotuning': No such file or directory

So that is not an availability test.

What is accessed by:

$ netsh interface tcp show global
Querying active state...

TCP Global Parameters
----------------------------------------------
Receive-Side Scaling State          : enabled
Chimney Offload State               : disabled
NetDMA State                        : disabled
Direct Cache Access (DCA)           : disabled
Receive Window Auto-Tuning Level    : normal
Add-On Congestion Control Provider  : none
ECN Capability                      : disabled
RFC 1323 Timestamps                 : disabled
Initial RTO                         : 3000
Receive Segment Coalescing State    : disabled
Non Sack Rtt Resiliency             : disabled
Max SYN Retransmissions             : 2
TCP Fast Open                       : enabled

For general Cygwin reference the following (elevated?) commands 
en-/disable WinHTTP RFC 1323 TCP Window Scaling: 

# netsh int tcp set global autotuninglevel=disabled

# netsh int tcp set global autotuninglevel=normal

Are there separate features and parameters for WinInet or other 
Windows flavours of network access, that affect their use of 
Window Scaling tweaks?

How does that interact with Bandwidth Reservation GPOs or reg entries: 
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\Psched == 0

And are there other settings, parameters, GPOs, or reg entries that 
also may affect the operation of TCP Window Scaling?

> 6.) I will consider looking into iperf3 once we have solved the 
> problem in 2. Also there would be a much greater chance that these 
> things get solved more quickly if we could solve the problem in
> number 4. One of my TCP devs likes cygwin and indicated a willingness
> to help. However, I will not allow any of my dev team to get mixed
> up with using @gmail accounts. Could cause too much trouble for the 
> youngster :)

HTH

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple