Speed problem

[uwp at dicke-aersche.de]

On Mon, 11 Nov 2002, Craig Barratt wrote:

> > > > You haven't really provided enough data to even guess what
> > > > is limiting your performance.
>
> How similar is the directory tree on the target (receiving)
> machine?  There are three general possibilities:
>
>   - It's empty.

That is the case at the beginning. Now there is some data.
But: Even when I'm doing scp directly from machine to machine
it never slows down that much, it holds a specific rate (between
15 and 18 MB/s, depends a little on other network traffic on the
cable). But it never drops 99%.

> In the first case rsync should be i/o limited (disk or network).

Let me tell you again: When I'm using every other program alone
this problem doesn't occur. It's just an rsync issue (ok, at this
time I can say rsync + ssh because I haven't tried rsh yet).

> In the second and third cases rsync could easily be cpu limited
> on the sending end.  In the third case it could also be disk
> (specifically seek) limited on the receiving end.  For example,

And again: Every other method of copying the data is quite normal.
The disk is not the problem. The network itself is not the problem.
Even the CPU is not the problem. The CPU gets slowed down a little
in the beginning as in every other ssh transfer too. But even this
is a linear process, it should never drop at such a huge amount.

> you might dump a large database to a binary file, whose content
> (records) are similar, but the order might change dramatically.
> This could take a huge number of seeks on the receiving machine
> to rebuild the file, even though only a small amount of data is
> transferred.

I'm transferring only 11 files, each about 60-70 GB and maybe 100
smaller ones.

> I'd try adding the -v option and see if the "slowdown" always
> happens on certain files.  Then try running rsync on just those

It happens on every file if it's big enough (it's a matter of time,
it seems that this effect is coming during a big transfer).
But ok, maybe the verbose mode will tell more, I'll try it.

> files.  If it is slow right away then maybe this explanation is
> correct.  If it still goes fast, then slows down, then there
> is something else going on.

Yes, it starts fast and drops after 5-10 minutes.

> As another test, run rsync to an empty target directory.  Rsync
> should be i/o limited for the entire running time.

How to limit i/o for rsync ?

Mermgfurt,
		Udo
-- 
Udo Wolter                      |  /"\
email:    uwp at dicke-aersche.de  |  \ / ASCII RIBBON CAMPAIGN
www:      www.dicke-aersche.de  |   X    AGAINST HTML MAIL
dark:    heaven at lutz-ziffer.de  |  / \