Why is my rsync transfer slow?

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

First, don't use -z on a local copy.  It will only make rsync slower
for no reason at all.

Second, 45 million files means 90 million calls to stat().  This will
take a while even if nothing needs copying.

On 01/21/2016 03:20 AM, dbonde+forum+rsync.lists.samba.org at gmail.com
wrote:
> I run a rsync job transferring about 45 million files/approximately
> 1.8 TB data (a Mac OS X Time Machine backup) over a 100 MBit
> connection.
> 
> I use rsync 3.1.1 from MacPorts (I first tried the built in rsync, 
> version 2.6.9, since it has a Mac OS X specific cache parameter,
> but it ran out of memory) with the following parameters
> 
> % rsync -HzvhErlptgoDW --stats --progress --out-format="%t %f %b" 
> /source/ /destination/
> 
> The source is an external 3.5" HDD connected with Firewire 800.
> The destination is a sparse disk image bundle mounted locally (but
> its "source file" is on a network storage). Initially I got good
> speeds, 7-9 MB/s for reasonably large files but the longer this
> operation has been going on (I restarted it three days ago, see
> below), the slower it gets. There are also long pauses when nothing
> happens, like this:
> 
> 2011-01-22-070305/Macintosh HD/Library/Application 
> Support/Apple/Mail/Stationery/Apple/Contents/Resources/Photos/Contents
/Resources/Bamboo.mailstationery/Contents/Resources/Mask3.png
>
> 
1.28K 100% 3.26kB/s 0:00:00 (xfr#48406, ir-chk=1050/4166332)
> 
> 2016/01/16 18:26:48 
> Volumes/src/Backups.backupdb/mm/2011-01-22-070305/Macintosh 
> HD/Library/Application 
> Support/Apple/Mail/Stationery/Apple/Contents/Resources/Photos/Contents
/Resources/Bamboo.mailstationery/Contents/Resources/Mask3.png
>
> 
313
> 
> 2011-01-22-070305/Macintosh HD/Library/Application 
> Support/Apple/Mail/Stationery/Apple/Contents/Resources/Photos/Contents
/Resources/Bamboo.mailstationery/Contents/Resources/banner-green.jpg
>
>  32.26K 100% 0.00kB/s 0:00:00 (xfr#48407, ir-chk=1049/4166332)
> 
> 2016/01/16 19:17:37 
> Volumes/2TB/Backups.backupdb/mm/2011-01-22-070305/Macintosh 
> HD/Library/Application 
> Support/Apple/Mail/Stationery/Apple/Contents/Resources/Photos/Contents
/Resources/Bamboo.mailstationery/Contents/Resources/banner-green.jpg
>
> 
31279
> 
> As you can see, the first file is finished 18:26, the second file
> 19:17, almost an hour for a file that is just 32 kB.
> 
> I don't think the transfer is CPU limited. There are some CPU
> spikes but generally CPU load is less than 10%. The three rsync
> processes spawned by this operation has, all in all, used almost
> exactly 5h of CPU time in the 72h the transfer has been going on.
> The computer itself idles 23h a day.
> 
> Nor is memory a problem. Memory pressure has been "green" since
> the operation begun.
> 
> Kernel task has accumulated quite a bit of CPU time (57h when I
> write this), but on the other hand, the uptime is 25 days and all
> these 57h can't have been consumed by rsync.
> 
> Some final details
> 
> * I had had this process running for a couple of days when I
> restarted it to get better logging three days ago. It took nine
> hours before the first file was transferred.
> 
> * I first used Finder to transfer this directory tree from the
> same source to the same destination. That took 3 days, all in all.
> Now I have spent 6 days and I don't think I even have transferred a
> third of the tree.
> 
> * I have tried transferring files between the same source and 
> destination outside of this operation and they go at full speed
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlag5JkACgkQVKC1jlbQAQcTwwCeKKbLa6UXxuiG7TJidqa1PKcT
lh0AnRfDtS90pUJFmDptXmyGEH09G0pS
=E+fZ
-----END PGP SIGNATURE-----