rsync speed problem when transferring bzip2 files

Justin Piszcz jpiszcz at lucidpixels.com
Tue Mar 30 15:49:53 MDT 2010


Hello,

$ rsync --version
rsync  version 3.0.7  protocol version 30

When I transfer files with rsync (bzip2 files specifically), I get 
6-11MiB/s on a 10Gbps network.

The same occurs on a gigabit network too incase you're wondering.

rsync:
r/1/b/l1/l1.root.ext4.dump.bz2
     21168128   0%    6.72MB/s    0:56:31
     28213248   0%    6.72MB/s    0:56:31

When I use FTP I get ~500MiB/s.

`r/1/b/l1/l1.root.ext4.dump.bz2' at 4816830464 (20%) 483.43M/s eta:37s [Receivi
`r/1/b/l1/l1.root.ext4.dump.bz2' at 5746065408 (24%) 494.15M/s eta:34s [Receivi
23362460206 bytes transferred in 46 seconds (487.76M/s)

What is the bottleneck problem here?  Why is it so slow?
Other files (non-bzip2) do appear to transfer > 100-200MiB/s.

Using rsync with rsh:
rsync --exclude=/proc/* --exclude=/sys/* --archive --update --verbose --stats --progress --hard-links --sparse --numeric-ids --skip-compress --delete --delete-excluded --delete-after -e rsh 10.0.0.254:/ ./l1
root     20819  0.4  0.0  12324   796 pts/18   S+   17:41   0:00 rsh 10.0.0.254rsync --server --sender -vulHogDtprSe.Lsf --skip-compress=--delete --numeric-ids . /

Justin.



More information about the rsync mailing list