file contents cause rsync to fail (with certains args and dir structure)

[Dave Gordon]

Quite strange at first sight that the failure should depend on the files containing NULs!

But I've reproduced it on both Ubuntu and OpenSUSE with d73762e "Preparing for release of 3.1.3".

The problem remains even if you drop the --checksum or --delay-updates options from the command line, but goes away if you don't use both "--sparse" and "--preallocate" together.

So it looks like a bad interaction of these options, probably with the sparse handling being affected by the NULs in the files.



HTH,

.Dave.



Sent using Zoho Mail






---- On Sun, 04 Mar 2018 14:57:24 +0100 xftroxgpx via rsync <rsync at lists.samba.org> wrote ----




script to reproduce: 

 

#!/bin/bash 

 

#tested to fail as below: ArchLinux's rsync-3.1.3-1-x86_64.pkg.tar.xz 

#tested to fail as below: ArchLinux's rsync-3.1.3pre1-1-x86_64.pkg.tar.xz 

#tested to work ok : ArchLinux's rsync-3.1.2-8-x86_64.pkg.tar.xz 

 

if test "$1" == "clean"; then 

 rm -vrf destdir sourcedir sourcedir2 sourcedir3 

 exit 0 

fi 

 

echo '!! test 1:' 

 

mkdir -p destdir 

mkdir -p sourcedir/a 

#one \0 followed by a non-\0 (so, using a space) required: 

echo -ne '\0 ' > sourcedir/a/b 

#non zero size file required: 

echo -ne 'c' > sourcedir/c 

echo 'sourcedir' >/tmp/filesfrom.lst.tmp 

rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ 

#rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir/a/b": No such file or directory (2) 

#rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3] 

 

# ^ this happens first and any subsequent times! 

 

echo '!! test 2:' 

 

mkdir -p sourcedir2/a 

#one \0 followed by a non-\0 (so, using an M) required: 

echo -ne '\0M' > sourcedir2/a/b 

#one \0 followed by a non-\0 (so, using an M) required: 

echo -ne '\0M' > sourcedir2/c 

 

#in order to see this error: (for file 'c') 

 

#non-zero file size required and it must be prefixed by '.' aka dot 

#echo -ne '1' > sourcedir2/.d 

#XXX: ^ (un)comment, don't forget to ./go clean afterwards, then ./go 

 

#otherwise, the error is for file "a/b" 

 

echo 'sourcedir2' >/tmp/filesfrom.lst.tmp 

rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ 

#rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir2/a/b": No such file or directory (2) 

#rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3pre1] 

 

echo '!! test 3:' #same as 2 but an extra file '.d' exists! 

 

mkdir -p sourcedir3/a 

#one \0 followed by a non-\0 (so, using an M) required: 

echo -ne '\0M' > sourcedir3/a/b 

#one \0 followed by a non-\0 (so, using an M) required: 

echo -ne '\0M' > sourcedir3/c 

 

#non-zero file size required and it must be prefixed by '.' aka dot 

echo -ne '1' > sourcedir3/.d 

echo 'sourcedir3' >/tmp/filesfrom.lst.tmp 

rsync --recursive --perms --checksum --delay-updates --numeric-ids --preallocate --sparse --files-from=/tmp/filesfrom.lst.tmp -- ./ ./destdir/ 

#rsync: write failed on "/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/destdir/sourcedir2/c": No such file or directory (2) 

#rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3] 

 

A pristine copy of this script can be found here: https://github.com/xftroxgpx/a3/blob/37ebff3e0fe9d294aeec899a082dc2c51c486eb4/system/Z575/OSes/3archlinux/on_baremetal/filesystem_now/archlinux/home/xftroxgpx/sandbox/rsync/nsfod_issues/try2/go 

 

Just run ./go to start all 3 tests. 

 

All 3 tests should fail with: 

ArchLinux's rsync-3.1.3-1-x86_64.pkg.tar.xz 

ArchLinux's rsync-3.1.3pre1-1-x86_64.pkg.tar.xz 

 

All 3 tests will succeed with ArchLinux's rsync-3.1.2-8-x86_64.pkg.tar.xz 

(aka rsync version 3.1.2 protocol version 31) 

 

 

Cheers! 

 

​Sent with ProtonMail Secure Email.​ 

 

 

 

-- 

Please use reply-all for most replies to avoid omitting the mailing list. 

To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync 

Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html 






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20180304/5f2c2f1b/attachment.html>