Large file - match process taking days
shachar at shemesh.biz
Wed Jul 30 14:27:01 GMT 2008
Rob Bosch wrote:
> The files are very similar, a maximum of about 5GB of data differences over
> 80GB. The CPU on both sides is low (3-5 percent) and the memory usage is
> low (11MB on the client, not sure on the server).
> Full rsync options are:
> -ruvvvvityz --partial --partial-dir=.rsync-partial --links --ignore-case
> --preallocate --ignore-errors --stats --del --block-size=1149728 -I
> I'm using the -I option to force a full sync since date/time changes on
> database files is not a reliable measure of changes.
> I'll try the block-size at 1638400 although I have not seen a big change in
> moving it from about 287000 (default square root) to 1149728.
You wouldn't. If CPU utilization is low, this is not the problem.
What about network utilization? What does ntop have to say?
What about disk utilization? I'm not sure what the best way to measure
it would be (though munin does a good job of it)
More information about the rsync