Performance problem

[jw schultz]

On Mon, Jan 12, 2004 at 06:22:04PM -0500, Andrew Boyce-Lewis wrote:
> Hi, I am running rsync version 2.5.7 (stock distro on redhat linux, ES
> and 9) to rsync a directory with ~300k files in it from a machine on a
> 10Mbit internet tap to a machine with a 100Mbit internet tap. The
> problem is that I am only getting about 500Kbps during the transfer. I
> have tested link speed by scping a file between the two systems,
> performance was approximately 7.5Mbit/sec.  Ping time between the two
> locations is about 2.5 ms (only 20 miles apart)
> 
> This is obviously a problem. I have also tried rsyncing just a few files

It is?  Rsync and scp do two completely different things and
their performance is not comparable.

> (say 400) to rule out problems related to huge file lists, however I had
> the same problem. 
> 
> Is this a known problem? Am I missing something really obvious?
> 
> I have tried testing drive performance in each machine and have found it
> to be acceptable. 

But how is the filesystem performance at doing what rsync
does to it?  Raw sequential throughput is irrelevant.

> The line that I am using to do the rsync is the following:
> 
> rsync -avvu --exclude-from=/root/sync_list.exclude --delete -e ssh
> --stats /data/ dest:/data/

Try --whole-file;  if that improves things you are disk bound.

Drop to just one -v; -vv is for debugging and adversely
affects network buffering.

It probably isn't your exclude file but a large exclude file
would add overhead.

2.5.7 is a security release with minimal changes from 2.5.6
which is almost a year old.  There have been a lot of
changes since then most of which are in 2.6.0.  Since 2.6.0
there have been a number of changes related to performance
including one that dramatically improves the network
performance.  All of this you would have known if you had
read the list archives or checked NEWS.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt