problems with large transfers form solaris 2.6

[Dave Dykstra]

On Fri, Sep 28, 2001 at 08:42:36AM +1000, Tomasz Ciolek wrote:
> On Thu, Sep 27, 2001 at 09:48:18AM -0500, Dave Dykstra wrote:
> > On Wed, Sep 26, 2001 at 10:46:51AM +1000, Tomasz Ciolek wrote:
> > > Hi all.
> > > 
> > > I have a large data set on solaris 2.6 (approx 16gb and about 900K files
> > > and rectory structures). Rsync is misbehaving in the most obnoxious way:
> > > 
> > > it is launched as such: rsync -e ssh -aLv --bwlimit=512 --timeout=300 <source> <dest>
> > > 
> > > A transfer starts, a list of files and directories is built, and data
> > > starts flowing. then for no apparent reason data stops flowing form the
> > > rsync on the solaris end the the connection times out after 5 minutes of no
> > > IO activity. When the timeout parameter is not there, the rsync will just
> > > stop and hang around for hours untill killed.
> > > 
> > > The only other this worth mentioning is that the solaris box is hidden
> > > behind a firewall and we use a ssh proxy (TIS plug proxy) to connect to the
> > > box.
> > > 
> > > Addibd many v's on the commandline produces a lot of debug but no real
> > > information as to whats going on. The data flow just stops and there is no
> > > error messages on either end in rsync output or logs... I am stumped... I
> > > wander is there is no known issues with globbing functions (if rsync uses
> > > them) on solaris or some such...
> > > 
> > > Any isnights or pointers would be appreceated.
> > 
> > What version of rsync is running on both sides, and what version of ssh are
> > you using?  There have been versions of both that have hanging problems,
> > and there are even patches available for the current version of rsync
> > although I'm not sure whether they're relevant over ssh.  People have
> > reported that the "-v" option makes things worse.  Using "netstat" on both
> > sides (and probably on your proxy machine too) may help to detect where
> > packets are piling up in queues.
>  Hi Dave,
>  
>  Rsync 2.4.6 on both ends.
>  Openssh 2.9p2 on the NT end (Cygwin build, install early 09/2001)
>  SSH 1.2.22 on the Solaris end (don't ask me why, it not my box, I cannot
> make upgrades as I please)
> 
>       As far as we can tell there is no packets in the queues on the
> proxy or the solaris box. Thats why its so wierd....
> 
> Regards
> Tomasz Ciolek


I'm not very familiar with NT problems.

There is a known hanging problem with the SSH 1.2.X series -- it needs to
be compiled without the "USE_PIPES" option, which was the default at least
on 1.2.27 and later.   1.2.22 is very old indeed, with more than one widely
published security hole, so maybe you can pursuade the owner to upgrade to
1.2.27 and make a few modifications.  There's also security patches for
1.2.27, have them check out
     http://www.core-sdi.com/advisories/ssh1_deattack.htm
     http://www.core-sdi.com/advisories/ssh1_sessionkey_recovery.htm

Perhaps you're running into the problem fixed by Wayne Davison's patches:
    http://www.clari.net/~wayne/rsync-nohang.patch
    http://www.clari.net/~wayne/rsync-nohang2.patch

- Dave Dykstra