Rsync between HPUX11.11 and Redhat7.3

[jw schultz]

On Wed, Feb 26, 2003 at 03:11:12PM +0200, Stanton Scott wrote:
> Cool, I am now running rsync 2.5.6 on both the client and
> server side unfortunaly still no luck. 
> 
> The average file sizes are around 100K to 20MB. There are
> the occasional large files, no larger than 700MB,
> approximate total size of rsync server directroy to be
> coppied 16GB.
> 

The low hanging fruit is out of the way now.
We are on a 100Mb or faster switched LAN, yes?

If the transfer rate has plummeted to approximately
5KiloBytes/sec something pathalogical must be going on.
I've never heard of performance drops down to that
magnitude.  --whole-file might avoid the pathology but i'd
be concerned that something is seriously wrong.

On the biggest of files (>100MB) are you seeing any
"redoing" messages?  If not i'd say we have a problem
outside of rsync.  Rsync is notorious for stressing the disk
and memory systems to the point of failure.

Check the system logs on both ends, perhaps there are disk
errors or filesystem problems.  I don't know the HPUX
specific issues well enough to address them.
You will of course have avoided PDC and HPT controllers.

Perhaps there is a networking issue.  Netstat might show
something.

Since it is only 16GB i'd suggest setting up another machine
to play alternate roles.  Make copies of the relevant trees
to it (using something other than rsync) and have it play
one role and then the other.


> On Tue, 25 Feb 2003 13:02:18 -0800
>  jw schultz <jw at pegasys.ws> wrote:
> > On Tue, Feb 25, 2003 at 04:57:48PM +0200, Stanton Scott
> > wrote:
> > > Hi,
> > > 
> > > The number of files we are trying to mirror is
> > > approximately: 100 000 and growing.
> > 
> > That isn't very many.  Your root filesystem probably has
> > more.  File count is not the problem.
> > 
> > > The version of RSYNC are as follows: 
> > > - On HPUX   : 2.5.6 (client)
> > > - On Redhat : 2.5.4 (server)
> > 
> > Upgrade.  Lots of fixes between 2.5.4 and 2.5.6.
> >  Correction
> > to above: Data volume could be pushing you over the edge
> > in 2.5.4.
> > 
> > > The kernel version of the Redhat server is:  2.4.18-3
> > > The amount of physical memory on the Redhat server is:
> > 512m
> > 
> > Should be plenty of memory, especially with only 100,000
> > files.  And the worst of the swap storms were fixed in
> > 2.4.18
> > 
> > > The rsync server and client do not appear to be under
> > > much of a load when performing this rsync operation.
> > > 
> > > I have tried using rsync without using compression,
> > > but unfortunaly this did not correct the problem.
> > > 
> > > I am incorrect in saying that rsync freezes, I have
> > > identified it does not actualy freeze it just slows
> > > down incredibly ( +- 5kb per second transfer speed),
> > > it just appeared to freeze.
> > > 
> > > With a smaler amount of files rsync works perfectly
> > > between my HP and Linux box.
> > > 
> > > Im not sure why rsync slows down to such a degree, do
> > > you think playing with the socket options could have a
> > > possitive affect ?
> > 
> > No.
> > 
> > The most likely culprit is the fact that you are
> > down-rev.
> > Get up to 2.5.6 and if the problem persists we'll revisit
> > it.
> > 
> > If the problem persists you should characterize file
> > sizes
> > and identify the largest file size.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt