Speeding up rsync
kmk at sanitarium.net
Tue Aug 24 21:40:19 UTC 2021
Most of the speedup is due to ssh being bound to a single CPU. You will
get a little more speed out of a single CPU system but the speedup is
unlikely to go past 2 in parallel. Even with that you may end up adding
more latency via disk seeks to accomplish much.
On 8/24/21 5:12 PM, hancooper wrote:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, August 24, 2021 8:25 PM, Kevin Korb via rsync <rsync at lists.samba.org> wrote:
>> In my experience backing up multiple hosts to a single host can speed
>> things up. Especially if using rsync over ssh with multiple CPUs on the
>> single host. You would need to do some experimentation to determine the
>> best number for your hardware and network. Also, if you exceed that
>> number you can put more stress on the backup machine lessening the
>> impact to the hosts being backed up.
>> On 8/24/21 3:58 PM, hancooper via rsync wrote:
>>> It makes sense to we to run multiple rsync commands to speed up a backup.
>>> At work, some have argued that if I sync the data all to the same host,
>>> there is no advantage
>>> in parallelization. Whether you sync 3x 1G in parallel or 1x 3G makes
>>> no noticeable difference.
>>> Speed in general does not increase by increasing the number of sync
>>> However, if one want to sync to multiple hosts, the effort may be
>>> What is the current advice about speeding up data transfer ?
>> Kevin Korb Phone: (407) 252-6853
> Hi Kevin. What about a single host to another with multiple CPU's ?
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: https://sanitarium.net/
PGP public key available on web site.
More information about the rsync