How to make generator and receiver processes strictly sequential?
tschoening at am-soft.de
Sun Nov 8 16:31:38 UTC 2020
I have one server hosting virtual machines and one older NAS Synology
DS1512+ used as backup target for those virtual machines. The NAS uses
BTRFS for its internal storage because I want to use compression and
at best make effecient use of snapshots.
When using "--inplace --whole-file" the backup succeeds in ~2 hours,
which is OK to fit into avarious historically grown time windows for
different backups onto the NAS and from the NAs to USB itself. When
using "--inplace" alone OTOH the necessary time increases to ~16
hours, which is unacceptable long currently.
The bottleneck seems to be the NAS: With "--whole-file" it simply
writes somewhat constantly with 100+ MiB/s, while without it reads
drop to some broad range of ~35 to at best 80/100 MiB/s. Those read
numbers are not too constant anymore as well.
Looking at HTOP on the NAS, I can see two RSYNC processes most of the
time concurrently reading subsequent files. After asking about this
behaviour on SU, it's most likely that those processes are the
generator calculating block-based checksums and the receiver itself.
It seems even though the amount of changed data on the files is pretty
low and therefore writes occur pretty seldom, the concurrent reads do
make overall performance worse.
So, is there some option to make generator and receiver on the backup
target strictly sequential?
Mit freundlichen Grüßen,
Thorsten Schöning E-Mail: Thorsten.Schoening at AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
More information about the rsync