[Bug 12732] hard links can cause rsync to block or to silently skip files

Kevin Korb kmk at sanitarium.net
Wed Apr 5 23:29:42 UTC 2017


Note that in all --*-dest options if the path is relative it is relative
to the target dir not the "$PWD".  I like to always use absolute paths
because of this.  But essentially, the command with the instances of
$PWD vs without them the paths aren't the same.  If you want all
relative to . paths it would be:
rsync -aHvv --compare-dest=../dstlt/. srclt/. baklt/.

Also, all these params that end with /. the . is kinda pointless despite
being points ;)

On 04/05/2017 07:19 PM, L A Walsh via rsync wrote:
> just subscribed for rsync-qa from bugzilla via rsync wrote:
>> Probably using different options? Can this be some sort of Heisenbug,
>> nobody can reproduce? Do the two sequences of shell commands work for
>> you as expected? Please note that both rsync commands in the mail
>> generated by bugzilla are split into two lines (each): Both rsync
>> commands should read
>>   
> ---
> Actually, I am reproducing your bug exactly as you entered
> it.
> And it is very sensitive to options.
> 
> When I take out the $PWD it works.
> 
> First your case:
> /tmp> rsync -aHvv --compare-dest=$PWD/dstlt/. $PWD/srclt/. $PWD/baklt/.
> |& tee testlt.log
> sending incremental file list
> delta-transmission disabled for local transfer or --whole-file
> ./
> a is uptodate
> d is uptodate
> b
> a => b
> /tmp> ll -i srclt dstlt baklt
> baklt:
> total 8
> 17305770 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 17305770 -rw-rw-r-- 2 2 Apr  5 15:46 b
> dstlt:
> total 8
> 8474888 -rw-rw-r-- 1 2 Apr  5 15:46 a
> 8474896 -rw-rw-r-- 1 2 Apr  5 15:46 d
> srclt:
> total 16
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 b
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 c
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 d
> ----
> Then w/o $PWD:
> /tmp> rsync -aHvv --compare-dest=dstlt/. srclt/. baklt/. |& tee testlt.log
> sending incremental file list
> --compare-dest arg does not exist: dstlt/.
> delta-transmission disabled for local transfer or --whole-file
> ./
> b
> d
> a => b
> c => d
> /tmp> ll -i srclt dstlt baklt
> baklt:
> total 16
> 17305770 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 17305770 -rw-rw-r-- 2 2 Apr  5 15:46 b
> 17305782 -rw-rw-r-- 2 2 Apr  5 15:46 c
> 17305782 -rw-rw-r-- 2 2 Apr  5 15:46 d
> dstlt:
> total 8
> 8474888 -rw-rw-r-- 1 2 Apr  5 15:46 a
> 8474896 -rw-rw-r-- 1 2 Apr  5 15:46 d
> srclt:
> total 16
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 b
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 c
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 d
> --------------
> 
> However, if "dstl" is a different file system, it works:
>>  rsync -aHv --compare-dest=~/dstlt/. $PWD/srclt/. $PWD/baklt/. |& tee 
> testlt.log  sending incremental file list
> --compare-dest arg does not exist: ~/dstlt/.
> ./
> b
> d
> a => b
> c => d
> 
> In my usage both the compare-dest & baklt were 'technically'
> different file systems (though the compare-dest was a snap
> shot).
> -----------
> ARG: if baklt is on a different FS, but src+dst are same:
> /tmp> rsync -aHv --compare-dest=$PWD/dstlt/. $PWD/srclt/. ~/baklt/. |&
> tee testlt.log  sending incremental file list
> ./
> b
> a => b
> /tmp> ll -i srclt dstlt ~/baklt
> /home/law/baklt:
> total 8
> 3677653445 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 3677653445 -rw-rw-r-- 2 2 Apr  5 15:46 b
> 
> dstlt:
> total 8
> 8474888 -rw-rw-r-- 1 2 Apr  5 15:46 a
> 8474896 -rw-rw-r-- 1 2 Apr  5 15:46 d
> 
> srclt:
> total 16
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 a
> 8866 -rw-rw-r-- 2 2 Apr  5 15:46 b
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 c
> 8867 -rw-rw-r-- 2 2 Apr  5 15:46 d
> 
> ----
> Ok... something's screwy here.  But no hangs.
> 
> Like I said in my 1st response, if some hard links
> were missing in the bak directory, I really would be
> unlikely to notice, as I didn't even try to check
> them (way too many files), but I never got a hang --
> and that's the part I'm thinking might be nfs
> related, since I've seen several issues with nfs not
> working the same way as a local fs.
> 
> I usually use smbfs -- in my usage, it's faster and I
> usually have fewer compatibility problems.
> (faster meaning ~ Reads~162MB/s, W~220MB/s,
> though i've seen explorer hit over 400MB/s).
> (using a 10Gb link).
> 
> But the above testing shows some unexplained behaviors
> out of rsync that sure look like a bug.
> 
> Good test case!
> :-)
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 224 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/rsync/attachments/20170405/06c98e13/signature.sig>


More information about the rsync mailing list