threaded rsync to improve performance over long distance ?

Joe josephj at main.nc.us
Sun Jan 20 15:39:55 MST 2013


At the risk of stating the obvious, is there any way you can break your
datasets into smaller pieces?  Then, you could start several rsync
processes in parallel - one for each piece, possibly using separate
Internet connections.

As Kevin says, I'm sure there is no direct relationship between rsync
and scp - they are two separate transfer "protocols", but the above
might get around whatever bottleneck you are running into.

If rsync is slow, then any other protocol (taking the same path) will
probably be just as slow.  You can't get a wider pipe by changing faucets.

I don't know much about networking, but if you could use network
diagnostic tools like ping and traceroute to find the actual cause(s) of
the bottleneck (without getting accused of security violations), then
you might be able to find a solution.  E.g. maybe you could arrange to
send your data somewhere closer first with better bandwidth and use
their connection (hopefully faster as well - with their
permission/service agreement) to send it the rest of the way.

Joe

On 01/20/2013 02:47 PM, Markus Moeller wrote:
> I try to rsync a lot of data over a long distance AU to UK and get a
> slow data rate because of the known slow scp performance over  long
> distances. Would it be possible with rsync to run multiple scp process
> at the same time ?
>
> Thank you
> Markus
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20130120/1bb91f44/attachment.html>


More information about the rsync mailing list