Parallel rsync
Matt McCutchen
matt at mattmccutchen.net
Thu Dec 11 02:58:34 GMT 2008
On Wed, 2008-12-10 at 12:10 -0800, Jim Searle wrote:
> Has there ever been a discussion about parallelizing rsync? We do
> some large file transfers across our wan which get done much faster if
> we run multiple rsync's at the same time. I wrote a perl script that
> traverses the directories and figures out how many files to send with
> each rsync process, but this only works properly for the initial copy.
>
> If we could say "rsync --parallel=8 ..." and as rsync finds a file to
> transfer it can open up new socket to do the transfer.
http://lists.samba.org/archive/rsync/2008-November/022187.html
For now, your script could run "rsync -ni" to find out which files need
to be copied without actually copying them and divide up the work based
on that information.
--
Matt
More information about the rsync
mailing list