rsync through high latency connection

[Dan Pritts]

there are also some issues with using ssh as your transport
that can slow down transfers over high-bandwidth links.

 http://www.psc.edu/networking/projects/hpn-ssh/

You can tune TCP as well - see www.web100.org for lots and lots
of info.  

Most of this stuff is written to deal with extremely high-bandwidth
transfers but the prinicples will probably still apply in your situation -
the ratio of bandwidth to latency is still the problem.  Most of us just
don't notice it on our 1.5Mb links.


On Sun, Nov 07, 2004 at 07:08:07AM +1300, Jason Haar wrote:
> Wayne Davison wrote:
> 
> >On Fri, Oct 29, 2004 at 07:03:43PM -0700, Rudy Moore wrote:
> > 
> >
> >>I noticed in the feature list that rsync pipelines file transfers to
> >>minimize latency - does this only affect transfer of large numbers of
> >>files?
> >>   
> >>
> >
> >It should be true of any transfer, but I'd estimate that the round-trip
> >time gets factored in twice per transfer, so it would get amortized the
> >more files you include in the transfer.
> >
> > 
> >
> 
> I don't think "pipeline" means anything special here: rsync treats TCP 
> transfers the same way FTP or HTTP does - it just lets TCP stream it as 
> fast as it can. (compare with NFS or SMB which turns the data into 
> "blocks" and doesn't allow TCP to stream it - resulting in TERRIBLE 
> throughput on high-latency links compared with HTTP/rsync)
> 
> >>Are there configuration steps that I can use to improve throughput?
> >>   
> >>
> High-latency means you'll never fill the pipe with a single 
> transmission. If you can, split the rsync job into sub-directories and 
> run separate rsync jobs on those at the same time. You might find (all 
> depends on your latency) that if you have a 1.5Mbs pipe, then you get a 
> max of 0.3Mbs for a single rsync job. But if you run 4 rsync jobs 
> simultaneously, you get 1.2Mbs. YMMV
> 
> Jason
> 
> -- 
> To unsubscribe or change options: 
> http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


danno
--
dan pritts - systems administrator - internet2
734/352-4953 office        734/834-7224 mobile