Why is my rsync transfer slow?

dbonde+forum+rsync.lists.samba.org at gmail.com dbonde+forum+rsync.lists.samba.org at gmail.com
Sat Jan 23 16:46:32 UTC 2016


On 2016-01-21 09:20, dbonde+forum+rsync.lists.samba.org at gmail.com wrote:
> I run a rsync job transferring about 45 million files/approximately 1.8
> TB data (a Mac OS X Time Machine backup) over a 100 MBit connection.
>
> I use rsync 3.1.1 from MacPorts (I first tried the built in rsync,
> version 2.6.9, since it has a Mac OS X specific cache parameter, but it
> ran out of memory) with the following parameters
>
> % rsync -HzvhErlptgoDW --stats --progress --out-format="%t %f %b"
> /source/ /destination/

Well, after some examination I found at least one problem with this 
transfer (that is still running): hard links are not preserved:

This is how a certain file looks at the source where it is backed up on 
several locations using hard links:

source volume:

zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG"
9236871 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG

zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG"
9236871 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG


destination volume:

zsh-% ls -i "/…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG"
20765913 /…/backups/2011-06-23-040258/Pictures/DSCF0748.JPG

zsh-% ls -i "/…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG"
704428 /…/backups/2010-12-18-070445/Pictures/DSCF0748.JPG

As you can see the inode number is the same on the source volume while 
it is completely different on the destination volume.

Why are my hard links not preserved? I thought the purpose with -H was 
to transfer the hard links rather than the file itself.



More information about the rsync mailing list