This is the mail archive of the cygwin-patches mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance


Hi,

> I'm not exactly concerned about Linux being way faster accessing an NTFS
> drive.  After all it's the OS itself and comes with it's own NTFS driver
> which obviously is streamlined for typical POSIX operations.

I did not test & compare to using the Linux NTFS, rather I compared with Linux on VMWARE using the same Windows NTFS.SYS (via the same kernel32.dll APIs):

Cygwin: "C:/cygwin/bin/ls.exe /bin" -> cygwin1.dll -> kernel32.dll -> NTOS kernel -> NTFS.SYS driver -> HD

linux: "/bin/ls /mnt/hgfs/C/cygwin/bin" -> glibc -> linux kernel -> VMWARE hgfs driver -> vmware_player.exe (on Win32) -> kernel32.dll -> NTOS kernel -> NTFS.SYS driver -> HD

As you can see the VMWARE path is much longer than Cygwin, and it passes the same APIs and NTFS.SYS driver, and yet it executes much faster.

This helps us understand that there is a lot that still can be done in Cygwin's filesystem performance.

Yoni


On 14/9/2010 11:38 AM, Corinna Vinschen wrote:
On Sep 13 13:45, Yoni Londner wrote:
Hi,

Abstract: I prepared a patch that improves Cygwin Filesystem
performance by x4 on Cygwin 1.7.5 (1.7.5 vanilla 530ms -->   1.7.5
patched 120ms). I ported the patch to 1.7.7, did tests, and found
out that 1.7.7 had a very serious 9x (!) performance degradation
from 1.7.5 (1.7.5 vanilla 530ms -->   1.7.7 vanilla 3900ms -->   1.7.7
patched 3500ms), which does makes this patch useless until the
performance degradation is fixed.

The problem is, I can't reproduce such a degradation. If I run sometimg like `time ls -l /bin> /dev/null', the times are very slightly better with 1.7.7 than with 1.7.5 (without caching effect 1200ms vs. 1500ms, with caching effect 500ms vs. 620ms on average). Starting with 1.7.6, Cygwin reused handles from symlink_info::check in stat() and other syscalls. If there is such degradation under some circumstances, I need a reproducible scenario, or at least some strace output which shows at which point this happens. Apart from actual patches this should be discussed on the cygwin-developer list.


First of all, even your results of 1200-1500ms (1st time) and 500-600ms (2nd time) is still way way way too long. On linux with an NTFS mount of C:/cygwin, this took<2ms!

And even on Win32 CMD.EXE this same operation will take you less
than 100ms. which is 5x to 10x faster.

The main reason for the difference: the Windows CMD.EXE does not
open file handles, which make the NTFS file system to actually go
and read each file's first 16KB of contents (even though you did not
ask for it!).

I'm not exactly concerned about Linux being way faster accessing an NTFS drive. After all it's the OS itself and comes with it's own NTFS driver which obviously is streamlined for typical POSIX operations.

And then there's Win32 which can go through a dir much faster as well,
since it doesn't have to care for POSIX compatibility of the result, and
the OS function calls coincidentally match what a cmd "dir" call needs.

If you're looking for a fair comparision, why don't you look for
Interix?  I did, and what I see is pretty much the same thing we do in
Cygwin.  Actually, with the last Cygwin from CVS an ls -lR on a
non-marginal directory tree is already faster than the same operation
under Interix.

That doesn't mean I won't look for more ways to enhance Cygwin's
performance, but it won't be by adding CYGWIN environment switches
or by neglecting correct information in stat.


Corinna




Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]