Native Parallelization in rsync

ameirh at ameirh at
Mon Sep 9 17:38:51 CEST 2013

It's great you brought up bbcp.  I didn't factor this into my initial email, but if we could further split up large files into N chunks and transport them concurrently, that would provide massive benefits as well.  

It's clear there are multiple areas for improvement that would let us do away with having to use other tools for various scenarios.  If we could put some planning/development time in even the easiest-to-implement cases, it would be highly-beneficial to a large number of users.

-AmeirFrom: Jason_Haar at<Jason_Haar at>
To: <rsync at>
Sent: Sunday, September 8, 2013
Subject: Re: Native Parallelization in rsync

On 08/09/13 18:28, Francis.Montagnac at wrote:
> There is another rather common case I think where this would help:
> when the destination directory is located on a NAS over NFS.

I think the biggest use-case is high bandwidth, high latency links. eg
you have a 50Mbs WAN link to a site in another country, and can never
get more than (say) 5Mbs. Parallelizing the data transfer could easily
push that up to 20-30Mbs

...and there is a "competitor" to rsync that does this - bbcp. Mirror a
directory from hostA to hostB using 'N' tcp streams. Runs like the
clappers :-)


Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options:
Before posting, read:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the rsync mailing list