Where is my bottleneck?

[Tim Conway]

This seems obvious, but it's easy to forget.  Are all the filesystems on 
both ends locally-attached?  If they're NFS or SMBFS, rsync is about a 
third the speed of SCP, unless you use "-W" to tell it to send the whole 
file if the timestamp/size don't match... in most cases, anyway.  ethernet 
lan and dialup wan to the remote server, I'd probably go ahead and let it 
use the rsync algorithm.

Try not using "-z"... Actually, maybe you already aren't.

Here's a strange one for you to try.
I'm sure you're running wide open.  Try using --bwlimit= to hold it back a 
bit.  If your network is chattering, backing off the throttle might get 
you down the track a hair faster.

Since everything is inside, try opening up rsh for a while (don't leave it 
open if you're internet-connected.)
on host1
time dd if=/dev/zero bs=1024k count=1024k |rsh host2 dd of=/dev/null
That'll give you a good idea of what your network can do.

Try tarring up whatever you're sending by rsync, sending it straight to 
/dev/null.
Feed it through dd for a byte count.
time tar -cf - whatever |dd of=/dev/null bs=1024k
Some controllers don't do very good high-speed/volume I/O.  Cache will 
keep things seeming fast, but when you go past cache, you see the true 
speed of your disk subsystem.  The above test will show you some of that.
if read's good, check write
time dd if=/dev/zero bs=1024k count=1024k 
of=/bigfilesystem/fileyouregoingtodelete

Tim Conway
Unix System Administration
Contractor - IBM Global Services
conway at us.ibm.com




I'm running rsync via ssh on two machines connected with 100 Mbit/s
Ethernet cards (at high speed) via a Linksys switch.  It's all right
here in a single closet.  I'm sending large files (initial transfer,
nothing preexists on the destination machine) and seeing transfer rates
in the 2 Mbit/s range.  The CPUs on both machines are more than 90%
idle, and I upped the --block-size to 256 kilobytes.  Disk activity is
very light.  Fedora Core 1, i686.

I don't expect 100 Mbit/s by any means, but 2?  Is this typical?  What
am I missing?