[Bug 12732] New: hard links can cause rsync to block or to silently skip files
L A Walsh
rsync at tlinx.org
Wed Apr 5 20:05:42 UTC 2017
just subscribed for rsync-qa from bugzilla via rsync wrote:
> Hard link handling seems to be broken when using "rsync -aH --compare-dest". I
> found two possible scenarios:
> 1) rsync completes without error message and exit code 0, although some files
> are missing from the backup
> 2) rsync blocks and must be interrupted/killed
> Further information
> This problem exists at least for rsync versions 3.1.0, 3.1.1, and 3.1.2 for
> different Linux varieties using various file systems:
I ran rsync 3.1.1 for over a year to help generate
snapshots. I can't say if it copied all the files or not, as
it was backing up a large "/home" partition, BUT, it never hung.
It did take 45min to a few hours to do the compare, but it
was comparing a large amount of data (>750G) w/a snapshot
(another 750G) to dump diffs to a third, and my /home partion
has a *very* large number of hard links.
So I know that hardlinks are handled 'fine' on comparing
'xfs' to 'xfs'.
> Latest test on openSUSE 42.2 (x86_64) on ext4 + on nfs with
Ah... I'd suspect nfs...
Why are you using nfs? rsync was designed to compare
against local file systems. You should try running rsync
directly from the nfs-host machine to the client and bypassing
NFS. I.e. -- you need to bypass NFS, since local->local
with hardlinks works.
Just checked my /home partition.
find shows 9295431 names (of any type), but du shows
(using du --inodes) shows 4407458 inodes. That means over
half of the filenames are hard linked. While my home
partition takes up 60% more space now, even cutting
those counts in half would still a large number of
hard links -- and rsync didn't crash doing an
rsync of the partition to an empty one, but first comparing
to a previous snapshot (the empty partition ended up
with differences between the main partition & the snapshot.
I'd remove NFS...
More information about the rsync