Native Parallelization in rsync

[ameirh at gmail.com]

I'm sure this is a topic that's come up plenty of times before, but is it possible to implement native parallelization in rsync?  There are several blog posts listing workarounds, but they are largely unideal.


My initial thought would be that rsync could split up the file list across N workers; the incremental file list would still be generated as it does today, but can then be divided evenly across a user-specified number of threads/processes that do the actual transfer of the files.  I can think of several tweaks to this process, but I think this is a good starting point.


I'm sure there are challenges with this approach that I'm not aware of, but it would be a great way to take advantage of free system resources and bandwidth, especially when transferring thousands of files.


Is this something that's in the works or that can be in the works?


Thanks for the awesome project!


-Ameir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20130904/d81ea992/attachment.html>