Rsync Optimization for Long Haul Links

[David Halliday]

Rsync speed seems to be effected more by latency than bandwidth, why is
this?

I have been happily rsyncing all over the world for some time, but today I
had cause to sync 40GB of data to all our remote sites causing me to look
closer at the statistics.

We effectively pull data using.

rsync -az --delete --force hqserver::filesystem /export/sync/filesystem

Our network is a 20Meg+ T3 at HQ and a T1/E1 at each remote site. All
running Hardware VPN over internet.

When rsync is the only network user I am only getting a fraction of the
available bandwith utilized. With the above command running at each site I
used 'show int' on the VPN router to get a breakdown of bandwidth use on a
site to site basis.

Total Outbound Usage 5354000 bits/sec (out of 20Meg) at California
>From CA to Arizona 1071000bits/sec (Out of 1.5Meg)
>From CA to Texas    838000bits/sec (Out of 1.5Meg)
>From CA to Boston   672000bits/sec (Out of 1.5Meg)
>From CA to France   429000bits/dec (Out of 2.0Meg)
>From CA to UK       312000bits/sec (Out of 2.0Meg)
>From CA to Germany  444000bits/dec (Out of 500K)
>From CA to Korea    452000bits/dec (Out of 1.5Meg)
>From CA to Taiwan   395000bits/dec (Out of 1.5Meg)
>From CA to Singapore 340000bits/dec (Out of unknown)

Finally, when I run several bandwidth using apps (such as ftp, scp etc)
concurrently I can get close to line speed at any site all sites.

>From this I conclude a single rsync at a site can not utilize all available
bandwidth. It is also interesting to note that the bandwidth the rsync is
able to utilize is directly proportional to the distance (latency) to the
remote site.

Now the question, Is there any way to optimize these transfers so that a
single rsync can use the entire bandwidth available?

I considered trying different block sizes but most references I found
suggested that I leave it to rsync to find the optimum size.

Dave.