File fragmentation

Rob Bosch robbosch at
Wed Sep 13 23:07:40 GMT 2006

my vote is for a command-line option.  I’ve noticed there is some
penalty for very large files (35GB-50GB).  The penalty is relatively small
based on my ‘intuitive’ measurements
read me watching without running a real
timer.  The difference is very small compared to what happens after a few
weeks without the fragmentation patch.  Our SAN was becoming so fragmented
that we were only getting 10% of normal throughput.  Since we’ve applied
this patch our SAN is continuously getting maximum throughput.  Benchmarks
are numbers such as 10MB/sec throughput due to fragmentation versus
90MB-150MB without fragmentation (numbers reported by telnet sessions to the
SAN device).

Fragmentation decreases are very significant.  We see most files now created
without any fragmentation (single fragment reported by sysinternals contig
utility) whereas prior to the patch the same files were containing 40,000
fragments and above.  In some cases we are still getting fragmented files
but those files normally contain less than 2000 fragments and are only on
files 10GB and above.

Just my two cents

More information about the rsync mailing list