Speed problem

[jw schultz]

On Tue, Nov 12, 2002 at 01:05:38AM +0100, uwp at dicke-aersche.de wrote:
> On Mon, 11 Nov 2002 jw schultz wrote:
> 
> > You haven't really provided enough data to even guess what
> > is limiting your performance.
> 
> As I said in the last mail: One limit for sure is ssh.

Yes, I saw that.  Some time after i had said so.

> But: with arcfour I'm getting 18 MB/s and that's where
> rsync is actually starting. It's just getting down and
> down and that's the strange point.

Yes that does sound strange.  I'll agree with you there.
It sounds like you are hitting either a resource constraint
or a scheduling issue.

> > You need look at CPU utilization, I/O load and network load
> 
> The CPU is at 100% for encryption reasons of ssh. But I/O
> is not very much.

What is the CPU load of rsync on the receiver?  That is
important.

> > Is the 18MB/s actual data transfer (just what has changed)
> > during actual transfer?  If so you may be up against the
> > limitation of your disk+file systems.
> 
> The disks have an upper limit of 52 MB/s (ext2) respectively
> 45 MB/s (ext3). It's an IDE RAID with 12 WD disks.
> 
> > My gut reaction is that 18MB/s is probably just short of the
> > best sustained throughput you can get out of your disk
> 
> Nope. The average of the disks over a long period is between
> 31 and 38 MB/s. I tested it without ssh.
> 
> Just to say it again: I wouldn't have a problem with 18 MB/s,
> that's what I expected. I just have a problem with the fact
> that it goes down to 400 KB/s in half an hour...:-(
> 

You are giving us dribs and drabs.  Now you mention ext2
and ext3 filesystems.  Since i'm not aware of any other OS
that uses them i'll assume you are running some form of
Linux.

I'm no performance tuning expert and you may wish to enquire
of Marcin, Peter, Rik and the linux-kernel gang.  They will
need to know what kernel(s) and patches you are running,
what network cards, how much memory, and things from vmstat,
iostat, netstat.

Frankly, the fact that it starts out running with near 
expected performance but proceedes to decay sounds less
like a rsync problem and more like something interesting
happening in the kernel.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt