rsync shows poor throughput vs. scp
tony at servacorp.com
Thu May 25 02:30:23 GMT 2006
You should get some better answers, but a couple of points jump out at me.
If the files are already compressed, "small" changes result in very
so the business of reading both the target and the source to find common
stuff is kinda counterproductive.
Also the -z (compressing something already compressed) can't really do much
particularly with LAN speeds
Methinks in this case, you want to disable a bunch of stuff that makes rsync
Basically, you want rsync to copy the entire file if the timestamp is
You do not want thingees that are extremely useful for transferring big
files over bad connections.
(I suspect that here is a case where you use rsync because it's easier to
figure out rsync
that to use something (actually quite different) that is more "appropriate".
You discover that "copy stuff from here to there (or vice-versa) is actually
incredibly complicated as to exactly what you (should) mean.
From: rsync-bounces+tony=servacorp.com at lists.samba.org
[mailto:rsync-bounces+tony=servacorp.com at lists.samba.org]On Behalf Of Marty
Sent: Wednesday, May 24, 2006 2:29 PM
To: rsync at lists.samba.org
Subject: rsync shows poor throughput vs. scp
hi all- been reading through the archives but I still can't seem to find a
solution to my problem. I am using rsync to keep mirror copies of content
which is being served (via http) on both the sender and receiver.
The files on average are 20-50mb each, (mostly already in a compressed
format... mp3, etc) and both sender and receiver pushes around 30mbps on
the sender is a Celeron 1.3GHz with 1GB RAM
receiver is an AMD 1.2GHz with 1GB RAM
Both have dual IDE drives configured for software RAID0
When I initiate an rsync from the receiver, the file list is built in
around / under 1 minute, and the transfer proceeds at around 500K/s
This seems very slow since the machines are on the same subnet and sitting
right next to each other in the same datacenter. While this rsync is
running, I open a second terminal and do an scp, again initiated from the
receiver copying from the sender, and the transfer proceeds at around 5MB/s!
Why is scp so much faster than rsync here? Is there anything I can do to
improve the speed of these transfers?
Fwiw, this is the rsync command I'm issuing:
rsync -azL --whole-file --stats --progress --delete sender::my_files
and rsyncd.conf on the sender looks like:
use chroot = no
max connections = 10
pid file = /var/run/rsyncd.pid
motd file = /etc/rsync/rsyncd.motd
timeout = 300
transfer logging = yes
log file = /var/log/rsyncd.log
log format = %t %h (%a) %m %l %b %o %f
uid = root
path = /my_files
max connections = 5
read only = true
hosts allow = 192.168.0.0/24
dont compress = *
list = false
MUCH respect and appreciation to anyone who can help!
-------------- next part --------------
HTML attachment scrubbed and removed
More information about the rsync