Why is my rsync transfer slow?

Kevin Korb kmk at sanitarium.net
Sat Jan 23 16:50:33 UTC 2016


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It will, assuming it sees both links in the same rsync run.

On 01/23/2016 11:46 AM, dbonde+forum+rsync.lists.samba.org at gmail.com
wrote:
> On 2016-01-21 09:20, dbonde+forum+rsync.lists.samba.org at gmail.com
> wrote:
>> I run a rsync job transferring about 45 million
>> files/approximately 1.8 TB data (a Mac OS X Time Machine backup)
>> over a 100 MBit connection.
>> 
>> I use rsync 3.1.1 from MacPorts (I first tried the built in
>> rsync, version 2.6.9, since it has a Mac OS X specific cache
>> parameter, but it ran out of memory) with the following
>> parameters
>> 
>> % rsync -HzvhErlptgoDW --stats --progress --out-format="%t %f
>> %b" /source/ /destination/
> 
> Well, after some examination I found at least one problem with
> this transfer (that is still running): hard links are not
> preserved:
> 
> This is how a certain file looks at the source where it is backed
> up on several locations using hard links:
> 
> source volume:
> 
> zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG" 
> 9236871 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG
> 
> zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG" 
> 9236871 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG
> 
> 
> destination volume:
> 
> zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG" 
> 20765913 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG
> 
> zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG" 
> 704428 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG
> 
> As you can see the inode number is the same on the source volume
> while it is completely different on the destination volume.
> 
> Why are my hard links not preserved? I thought the purpose with -H
> was to transfer the hard links rather than the file itself.
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlajr1kACgkQVKC1jlbQAQdG3QCgwRt/K9u6xrxGFeZP2uoPoaoT
OlcAnjE4eozRjJ1Mb9YC88YNhVTLEpP8
=p3pD
-----END PGP SIGNATURE-----



More information about the rsync mailing list