Help -- rsync Causing High Load Averages

[Wayne Davison]

On Tue, Mar 28, 2006 at 04:58:42PM -0800, Plugger wrote:
> When it runs, however, the load averages on the content1 server
> continue to grow to the 100s, bringing the server to a practical
> standstill. After a day of tweaking, and killing the content1 server
> periodically, I have not been able to reduce the loads much.

You should first determine what lacking resource is causing the high
load average.  If the CPU is maxed out, the decision to "nice" rsync is
a good one, but also consider using the --whole-file option (it will use
less CPU on the sender, and is a good choice for those with a fast
network connection and too little CPU).  If it is not CPU, it is
probably disk I/O, which rsync doesn't have a very good way to ration --
you can try using --bwlimit (which doesn't really help much during the
filesystem scan) or try the slow-down.diff in the patches dir (which is
an overly-simple sleep idiom that only affects the filesystem scan), but
neither one is really the right solution when rsync is using too much
I/O (and the OS doesn't have a "nice" equivalent for prioritizing
multiple process's access to I/O).

..wayne..