This is the mail archive of the
cygwin
mailing list for the Cygwin project.
Re: slow handling of large sets of files?
On Tuesday, February 01, 2005 7:53 PM [EST], Brian Dessent wrote:
> Ken Sheldon wrote:
>
>> Not only Cygwin apps incur this large performance penalty.
>> Something similar happens with the cmd.exe prompt command "DIR",
>> with the windows file explorer, or with IIS (FTP server). This
>> only seems to happen in the directory structures created by my
>> CygWin scripts (using apps: tar, wget, cp)
>
> If I had to take a wild guess I'd say that's because when a normal
> win32 app creates a file it usually inherits its ACL from the parent
> directory, and presumably NTFS is tuned for this in some way so that
> checking thousands of such files that all inherit ACLs is fast.
> However, when cygwin creates a file or dir it usually sets the
> permissions explicitly to match what you would expect on a POSIX
> system (i.e. 644 or 755.) Try creating your files/folders with
> "nontsec" set and see if the cygwin-created trees are as slow as
> native-created ones.
>
> Brian
NTFS in general just isn't very fast when you really start using some
of the advanced features. Its just not tuned very well for large
amounts of files. A perfect example of this is when you need to
delete alot of files and directories all at once - it takes an awfully
long time to do when compared to ext2/3 or reiserfs, and I've noticed
that sometimes the system doesn't properly clean up the directory
indexes, leaving stale ACL entries and such (run chkdsk, boot, do a
bunch of filesystem operations such as deleting files, then run chkdsk
again, and most likely, it will report even more errors).
MFT fragmentation seems to compound already existing problems.
--
Brian Bruns
The Summit Open Source Development Group
http://www.sosdg.org / http://www.ahbl.org
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/