[Bug 12732] New: hard links can cause rsync to block or to silently skip files

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=12732

            Bug ID: 12732
           Summary: hard links can cause rsync to block or to silently
                    skip files
           Product: rsync
           Version: 3.1.2
          Hardware: x64
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayned at samba.org
          Reporter: samba at hlipp.de
        QA Contact: rsync-qa at samba.org

Overview
========

Hard link handling seems to be broken when using "rsync -aH --compare-dest". I
found two possible scenarios:

1) rsync completes without error message and exit code 0, although some files
are missing from the backup
2) rsync blocks and must be interrupted/killed

I found this bug when tracking down random hangs of rsync. It turned out that
most, but not all of these hangs occur when hard links are present. Therefore,
I hope the latter case might give some hints to a larger problem that might be
triggered by this hard link bug.


How to reproduce (1)
====================

[ Using Linux on e.g. ext4 ]
############################
mkdir srclt
cd srclt
echo x > a
ln a b
echo x > c
ln c d
cd ..

cp -aix srclt dstlt
rm dstlt/{b,c}

mkdir baklt

rsync -aHvv --compare-dest=$PWD/dstlt/. $PWD/srclt/. $PWD/baklt/. >> testlt.log
2>&1
############################


Actual Results (1)
==================

cat testlt.log
#####
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
./
a is uptodate
d is uptodate
b
a => b
total: matches=0  hash_hits=0  false_alarms=0 data=2

sent 173 bytes  received 160 bytes  666.00 bytes/sec
total size is 8  speedup is 0.02
#####
ls -il srclt dstlt baklt
#####
baklt:
total 8
3249818642 -rw-r--r-- 2 X X 2 2017-04-04 X a
3249818642 -rw-r--r-- 2 X X 2 2017-04-04 X b

dstlt:
total 8
2205741698 -rw-r--r-- 1 X X 2 2017-04-04 X a
2205741699 -rw-r--r-- 1 X X 2 2017-04-04 X d

srclt:
total 16
1138988347 -rw-r--r-- 2 X X 2 2017-04-04 X a
1138988347 -rw-r--r-- 2 X X 2 2017-04-04 X b
1138988348 -rw-r--r-- 2 X X 2 2017-04-04 X c
1138988348 -rw-r--r-- 2 X X 2 2017-04-04 X d
#####


Expected Results (1)
====================

The directory baklt should contain the entries b and c.
Entry c is completely ignored and does not show up in the log.
Entry a does not need to appear in the backup, as it is present in both
directories srclt and dstlt.


How to reproduce (2)
====================

[ Using Linux on e.g. ext4 ]
############################
mkdir srclt2
cd srclt2
echo x > a
ln a b
cd ..

cp -aix srclt2 dstlt2
rm dstlt2/b

mkdir baklt2

rsync -aHvv --compare-dest=$PWD/dstlt2/. $PWD/srclt2/. $PWD/baklt2/. >>
testlt2.log 2>&1
############################


Actual Results (2)
==================

=> rsync hangs and must be interrupted/killed.

cat testlt2.log
#####
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
./
a is uptodate
b
a => b
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(636)
[sender=3.1.2]
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at io.c(504)
[generator=3.1.2]
#####
ls -il srclt2 dstlt2 baklt2
#####
baklt2:
total 8
2191211 -rw-r--r-- 2 X X 2 2017-04-04 X a
2191211 -rw-r--r-- 2 X X 2 2017-04-04 X b

dstlt2:
total 4
2191208 -rw-r--r-- 1 X X 2 2017-04-04 X a

srclt2:
total 8
2191206 -rw-r--r-- 2 X X 2 2017-04-04 X a
2191206 -rw-r--r-- 2 X X 2 2017-04-04 X b
#####


Expected Results (2)
====================

rsync should not block.
Entry a does not need to appear in the backup, as it is present in both
directories srclt and dstlt.


Further information
===================

This problem exists at least for rsync versions 3.1.0, 3.1.1, and 3.1.2 for
different Linux varieties using various file systems:
https://lists.samba.org/archive/rsync/2015-April/030092.html

Latest test on openSUSE 42.2 (x86_64) on ext4 + on nfs with
rsync --version
#####
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc
#####

-- 
You are receiving this mail because:
You are the QA Contact for the bug.