reducing I/O and system load
elk at elk.ru
Fri Mar 3 08:11:55 GMT 2006
>>> > I'm using rsync for simple daily back up data from one HD to another. It
>>> > takes about 10 minutes daily under "nice -n 19". The problem is
>>> --bwlimit=KBPS limit I/O bandwidth; KBytes per second
MM>> --bwlimit only limits the rate of transfer over the network. Vilius
MM>> Puidokas encountered the same problem and traced it to the scanning as
MM>> rsync generates the file list. He wrote a patch that adds a --slow-down
MM>> option that makes rsync wait a few microseconds in between scanning each
MM>> file. Find it here:
DS> usleep(100) in that place of the source shown on the page really helps,
DS> but it dramatically increases the time of rsync'ing. I guess Vilius'
Just for information.
Not completely sure, but it seems that making huge rsync'ed
partitions "noatime" (/etc/fstab) slightly helps as well.
I guess that 50% or so of rsync usage is disk-to-disk backup,
strange that it's impossible to find anywhere the proper
solution on how to reduce I/O load. The workaround SHOULD
be suggested by the software directly, I believe...
More information about the rsync