How to speed up rsync when haveing lots of files
lbutler+rsync at covisp.net
Wed Mar 4 06:46:47 GMT 2009
On 3-Mar-2009, at 22:55, Daniel.Li wrote:
> -z is apparently affect the performance when CPU has a lower
> like 200MHz or so. When doing rsync, 100% cpu occupied, which limits
> network performance.
You should not use -z at ALL with large numbers of small files. The
increased latency of the compression/decompression will far exceed any
time saved in transmission. You're not on a 300 baud modem, I assume?
-z should be used when you are sending large files that are
compressible in the first place (so not video or mp3s, for example),
but it it only going to hurt you on small files.
How small? Depends on your connection. My rule of thumb, based on
nothing at all, is that the speed in megabits of my connection is the
size in megabytes that I start worrying about compression.
Compressible data over 15MB in size? OK, I will start to think about
using -z. If I was on a T1? 1.5MB. ADSL? 700K. Arbitrary, perhaps,
but it seems to have served me well. In fact, I found removing -z sped
up transfers quite a bit on my LAN.
> BTW: I don't know more about "--compress-level=NUM", how many levels
> could use to set?
See above. As for the number range, I've always assume a 0-9 range
much like gzip, though that is not explicitly given in the man page.
The quality of our thoughts and ideas can only be as good as the quality
of our language.
More information about the rsync