[Bug 12732] New: hard links can cause rsync to block or to silently skip files

L A Walsh rsync at tlinx.org
Wed Apr 5 20:05:42 UTC 2017


just subscribed for rsync-qa from bugzilla via rsync wrote:
> Hard link handling seems to be broken when using "rsync -aH --compare-dest". I
> found two possible scenarios:
>
> 1) rsync completes without error message and exit code 0, although some files
> are missing from the backup
> 2) rsync blocks and must be interrupted/killed
> ....
>
> Further information
> ===================
>
> This problem exists at least for rsync versions 3.1.0, 3.1.1, and 3.1.2 for
> different Linux varieties using various file systems:
> https://lists.samba.org/archive/rsync/2015-April/030092.html
>   
---

    I ran rsync 3.1.1 for over a year to help generate
snapshots.  I can't say if it copied all the files or not, as
it was backing up a large "/home" partition, BUT, it never hung.
It did take 45min to a few hours to do the compare, but it
was comparing a large amount of data (>750G) w/a snapshot
(another 750G) to dump diffs to a third, and my /home partion
has a *very* large number of hard links.

    So I know that hardlinks are handled 'fine' on comparing
'xfs' to 'xfs'. 
> Latest test on openSUSE 42.2 (x86_64) on ext4 + on nfs with
>   
----
    Ah...  I'd suspect nfs...
   
    Why are you using nfs?  rsync was designed to compare
against local file systems.  You should try running rsync
directly from the nfs-host machine to the client and bypassing
NFS.  I.e. -- you need to bypass NFS, since local->local
with hardlinks works.

    Just checked my /home partition.
    find shows 9295431 names (of any type), but du shows
(using du --inodes) shows 4407458 inodes.  That means over
half of the filenames are hard linked.  While my home
partition takes up 60% more space now, even cutting
those counts in half would still a large number of
hard links -- and rsync didn't crash doing an
rsync of the partition to an empty one, but first comparing
to a previous snapshot (the empty partition ended up
with differences between the main partition & the snapshot.

    I'd remove NFS...






More information about the rsync mailing list