Parallel rsync

Matt McCutchen matt at mattmccutchen.net
Thu Dec 11 02:58:34 GMT 2008


On Wed, 2008-12-10 at 12:10 -0800, Jim Searle wrote:
> Has there ever been a discussion about parallelizing rsync?  We do
> some large file transfers across our wan which get done much faster if
> we run multiple rsync's at the same time.  I wrote a perl script that
> traverses the directories and figures out how many files to send with
> each rsync process, but this only works properly for the initial copy.
> 
> If we could say "rsync --parallel=8 ..." and as rsync finds a file to
> transfer it can open up new socket to do the transfer.

http://lists.samba.org/archive/rsync/2008-November/022187.html

For now, your script could run "rsync -ni" to find out which files need
to be copied without actually copying them and divide up the work based
on that information.

-- 
Matt



More information about the rsync mailing list