rsync of big image files from virtual hosts takes lot of time

[Kevin Korb]

First, add --stats to find out how much is being transferred.  Second,
if these are fixed size or sparse image files then --sparse will be a
big help.  Finally, if they are not fixed size or sparse then --inplace
is probably making things worse.

--no-whole-file is the default btw unless you are doing a local copy.
If you are doing a local copy then just use cp as it is much faster at that.

On 10/20/2018 12:36 PM, Lentes, Bernd via rsync wrote:
> Hi,
> 
> i'd like to rsync some image files from virtual hosts each night. The image files are between 50GB and 400GB big.
> I thought rsync would be the appropriate solution because it just transfers the differences, not the whole file.
> But nevertheless rsync takes hours.
> The images aren't much modified over the day, i would expect just some tens of megabyte needed to be transfered.
> 
> I have the following command line:
> rsync -av --no-whole-file --inplace source destination.
> 
> Is that ok ? Any suggestions ?
> 
> I skipped the check with the hash values before the transfer because creating a hash value from a 400GB file takes loooong.
> But the man pages mentione that nevertheless AFTER the transfer a hash value is created on both sides to check that the files are equal.
> Is it possible to switch the after-transfer check off ?
> 
> I have SLES 12 SP3 and rsync 3.1.0
> 
> Thanks.
> 
> Bernd
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/rsync/attachments/20181020/5f5a536b/signature.sig>