On 12/12/2012 08:39 AM, Ryan Johnson wrote:
Does gcc/ld/whatever know the final file size before the first write?
No, but does it need to? posix_fallocate() does not change file
contents; it merely says that anywhere there was previously a hole must
now be guaranteed to be backed by disk. So gcc would write the file as
usual, and then just before close()ing the fd, do a final
posix_fallocate(fd, 0, len) with len determined by the final file size.
You have to posix_fallocate the entire file before any write that might
create a hole, because the sparse flag poisons the loader,
Is there really a flag stuck into the file when it becomes sparse?
and persists
even if all gaps are later filled. For example, if I invoke the
following commands:
cp --sparse=always $(which emacs-nox) sparse
cp --sparse=never $(which emacs-nox) dense
for f in sparse dense; do echo $f; time ./$f -Q --batch --eval
'(kill-emacs)'; done
cp --sparse=never dense sparse
for f in sparse dense; do echo $f; time ./$f -Q --batch --eval
'(kill-emacs)'; done
du dense sparse
This doesn't point to a flag in the file, so much as cached information
(the file system is remembering that 'sparse' used to be sparse, even if
it is no longer sparse). But your point about a file being cached at
some point while it is sparse, even if it is later made non-sparse, is
interesting.
The relevant output is:
sparse
real 0m1.791s
dense
real 0m0.606s
sparse
real 0m3.158s
dense
real 0m0.081s
16728 dense
16768 sparse
Given that we're talking about cygwin-specific patches for emacs and
binutils anyway, would it be better to add a cygwin-specific fcntl call
that clears the file's sparse flag?
What flag is there to clear? Your cp demonstration showed that even
when we do a byte-for-byte copy of every byte (and the file is
non-sparse), the file system cache remembers that it used to be sparse.
How do we defeat that file system cache?