Fragmentation on XFS

Jamie Lokier jamie at shareable.org
Mon Feb 25 21:46:23 GMT 2008


Rob Bosch wrote:
> > Was that simply due to writing too-small block to NTFS?  In other
> > words, would increasing the size of write() calls have fixed it
> > instead, without leaving allocated but unused disk space in the case
> > of a user-abort with --partial, --partial-dir or --inplace?
> 
> It could have been a function of the block size but I don't think so.  I
> never tested the strategies you list.  Under cygwin the posix_fallocate
> function is extremely efficient in that it immediately allocates the files
> and does no writing, yet still provides a single-extent file if it can be
> provided (just like fallocate if supported in the kernel).  Given that
> solved the problem I didn't pursue any other alternatives.

I'm thinking that if rsync is aborted after it calls posix_fallocate,
you might have some large amount of disk space used, but not indicated
in the file size.  That seems like a bad state to leave a filesystem
in, because it's mostly invisible.

That means the prealloc option should be off by default (on all systems).

I'm thinking if large writes would fix the NTFS problem, they wouldn't
leave the filesystem like that following an abort, so that could be
safely turned on by default, and everyone would benefit from improved
NTFS performance.

Another possibility is to incrementally preallocate large chunks,
e.g. 1MiB at a time while writing.  That wouldn't leave huge
preallocated space, and should improve NTFS performance enough, so
could be turned on by default perhaps.

-- Jamie


More information about the rsync mailing list