rsync of big image files from virtual hosts takes lot of time

[Kevin Korb]

Your rsync must be a bit old.  The current --stats output is about twice
as long as that.

Anyway, there isn't that much change in the file.  If you were using
rsync over ssh it would probably take around 30 minutes.  I would guess
cp would take longer but hard to say how much longer but it would be
less than your rsync is taking.

All that time is probably being wasted on rsync reading the target file
over the network to find out what is different about it.

On 10/21/2018 08:35 AM, Lentes, Bernd via rsync wrote:
> 
> 
> ----- Am 20. Okt 2018 um 20:05 schrieb rsync ML rsync at lists.samba.org:
> 
>> I don't see any --stats output.  It will tell you exactly how much data
>> was involved.
>>
>> A local copy means that rsync isn't networking.  Rsync speeds things up
>> by running on 2 separate computers and communicating using the rsync
>> protocol.  If you are running rsync on a single computer with the source
>> being a mounted SAN volume and the target being a mounted cifs share
>> then as far as rsync is concerned that is a local copy.
>>
>> So, either get the cifs server to speak rsync (either over ssh or with
>> rsyncd) or just use cp.  Only copying the changes isn't all that helpful
>> when rsync has to read the whole target file over the network just to
>> find out what is different about it.  That is why --whole-file is
>> implied when rsync isn't networking.
>>
>> On 10/20/2018 02:00 PM, Lentes, Bernd via rsync wrote:
> 
> This is the output with --stats for a 50GB image.
> Appended also the output from time.
> You see it's horrible slow.
> I will try cp.
> 
> Literal data: 187,432,960 bytes
> Matched data: 53,499,658,240 bytes
> File list size: 0
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 189,111,658
> Total bytes received: 3,277,035
> 
> sent 189,111,658 bytes  received 3,277,035 bytes  6,493.15 bytes/sec
> total size is 53,687,091,200  speedup is 279.06
> 
> real    493m49.055s
> user    15m28.933s
> sys     26m13.445s
> 
> Bernd
> 
>  
> 
> Helmholtz Zentrum Muenchen
> Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
> Ingolstaedter Landstr. 1
> 85764 Neuherberg
> www.helmholtz-muenchen.de
> Aufsichtsratsvorsitzende: NN
> Stellv.Aufsichtsratsvorsitzender: MinDirig. Dr. Manfred Wolter
> Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen
> Registergericht: Amtsgericht Muenchen HRB 6466
> USt-IdNr: DE 129521671
> 
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/rsync/attachments/20181021/3a522821/signature.sig>