Large file - match process taking days

[Shachar Shemesh]

Rob Bosch wrote:
> The files are very similar, a maximum of about 5GB of data differences over
> 80GB.  The CPU on both sides is low (3-5 percent) and the memory usage is
> low (11MB on the client, not sure on the server).
>
> Full rsync options are:
>
> -ruvvvvityz --partial --partial-dir=.rsync-partial --links --ignore-case
> --preallocate --ignore-errors --stats --del --block-size=1149728 -I 
>
> I'm using the -I option to force a full sync since date/time changes on
> database files is not a reliable measure of changes.
>
> I'll try the block-size at 1638400 although I have not seen a big change in
> moving it from about 287000 (default square root) to 1149728.
>   
You wouldn't. If CPU utilization is low, this is not the problem.

What about network utilization? What does ntop have to say?
What about disk utilization? I'm not sure what the best way to measure 
it would be (though munin does a good job of it)

Shachar