rsync through high latency connection

[Jason Haar]

Wayne Davison wrote:

>On Fri, Oct 29, 2004 at 07:03:43PM -0700, Rudy Moore wrote:
>  
>
>>I noticed in the feature list that rsync pipelines file transfers to
>>minimize latency - does this only affect transfer of large numbers of
>>files?
>>    
>>
>
>It should be true of any transfer, but I'd estimate that the round-trip
>time gets factored in twice per transfer, so it would get amortized the
>more files you include in the transfer.
>
>  
>

I don't think "pipeline" means anything special here: rsync treats TCP 
transfers the same way FTP or HTTP does - it just lets TCP stream it as 
fast as it can. (compare with NFS or SMB which turns the data into 
"blocks" and doesn't allow TCP to stream it - resulting in TERRIBLE 
throughput on high-latency links compared with HTTP/rsync)

>>Are there configuration steps that I can use to improve throughput?
>>    
>>
High-latency means you'll never fill the pipe with a single 
transmission. If you can, split the rsync job into sub-directories and 
run separate rsync jobs on those at the same time. You might find (all 
depends on your latency) that if you have a 1.5Mbs pipe, then you get a 
max of 0.3Mbs for a single rsync job. But if you run 4 rsync jobs 
simultaneously, you get 1.2Mbs. YMMV

Jason