rsync of big image files from virtual hosts takes lot of time
Lentes, Bernd
bernd.lentes at helmholtz-muenchen.de
Sat Oct 20 16:36:21 UTC 2018
Hi,
i'd like to rsync some image files from virtual hosts each night. The image files are between 50GB and 400GB big.
I thought rsync would be the appropriate solution because it just transfers the differences, not the whole file.
But nevertheless rsync takes hours.
The images aren't much modified over the day, i would expect just some tens of megabyte needed to be transfered.
I have the following command line:
rsync -av --no-whole-file --inplace source destination.
Is that ok ? Any suggestions ?
I skipped the check with the hash values before the transfer because creating a hash value from a 400GB file takes loooong.
But the man pages mentione that nevertheless AFTER the transfer a hash value is created on both sides to check that the files are equal.
Is it possible to switch the after-transfer check off ?
I have SLES 12 SP3 and rsync 3.1.0
Thanks.
Bernd
--
Bernd Lentes
Systemadministration
Institut für Entwicklungsgenetik
Gebäude 35.34 - Raum 208
HelmholtzZentrum münchen
[ mailto:bernd.lentes at helmholtz-muenchen.de | bernd.lentes at helmholtz-muenchen.de ]
phone: +49 89 3187 1241
fax: +49 89 3187 2294
[ http://www.helmholtz-muenchen.de/idg | http://www.helmholtz-muenchen.de/idg ]
wer Fehler macht kann etwas lernen
wer nichts macht kann auch nichts lernen
Helmholtz Zentrum Muenchen
Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
Ingolstaedter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: NN
Stellv.Aufsichtsratsvorsitzender: MinDirig. Dr. Manfred Wolter
Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen
Registergericht: Amtsgericht Muenchen HRB 6466
USt-IdNr: DE 129521671
More information about the rsync
mailing list