Delta-compression and --write-devices

Laurens Blankers laurens at blankersfamily.com
Wed Jun 10 17:52:36 UTC 2020


Hello,

This is my first time posting to this list. I think I might have found a 
bug, but I am not sure, so I appreciate any thought you may have.

I am running rsync 3.2.0dev from Git master revision 916faecb with the 
copy-devices patch from rsync-patches master revision f22e220b on Debian 
10 and I used the Debian build system to build rsync.

I am running the following command:

alpha# rsync -vvvvv --checksum --inplace --progress -- copy-devices 
--write-devices omega:/dev/dm-9 /dev/dm-7

In order to transfer an entire LVM2 partition from omega to alpha.

The transfer works fine when doing it initially. However when running it 
for the second time, instead of applying delta-compression, rsync will 
copy the entire device once again. If attempt to do the same but with a 
file as destination:

alpha# rsync -vvvvv --checksum --inplace --progress -- copy-devices 
--write-devices omega:/dev/dm-9 /root/file

The delta-compression does work. I tried initiating rsync from the 
sender instead of the receiver, but the behaviour is the same.

Based on the log output and the source code I suspect that the problem 
occurs because the delta-compression determines the size of the 
destination to be 0. I base that on the following diff of the rsync output:

recv_file_list done
-get_local_name count=1 /dev/dm-7
+get_local_name count=1 /root/file
-[Receiver] change_dir(/dev)
+[Receiver] change_dir(/root)
-generator starting pid=2490
+generator starting pid=2486
delta-transmission enabled
-recv_generator(dm-7,0)
+recv_generator(file,0)
-gen mapped dm-7 of size 0 ***
+gen mapped file of size 64400243
generating and sending sums for 0
+count=8026 rem=7643 blength=8024 s2length=3 flength=64400243 %%%
recv_files(1) starting

The line I marked with *** seems to indicate the size of dm-7 is 0. The 
line marked with %%% seems to indicate that in case of the destination 
being a file hashes are produced, but that lack of this line in the 
output when the destination is a device seems to indicate no hashes are 
produced.

I took a look at the code and it seems that stat’ing a device returns 
size 0 on Linux/Matt’s computer. There is code in receiver.c to 
compensate for this:

/* On Linux systems (at least), st_size is typically 0 for devices.
* If so, try to determine the actual device size. */
if (fd1 != -1 && IS_DEVICE(st.st_mode) && st.st_size == 0) {
OFF_T off = lseek(fd1, 0, SEEK_END);
if (off == (OFF_T) -1)
rsyserr(FERROR, errno, "failed to seek to end of %s to determine size", 
fname);
else {
st.st_size = off;
off = lseek(fd1, 0, SEEK_SET);
if (off != 0)
rsyserr(FERROR, errno, "failed to seek back to beginning of %s to read 
it", fname);
}
}

However this code seems to be missing in generator.c.

Any help on solving this or pointers how to properly file this as a bug 
are appreciated.

Sincerely,

Laurens




More information about the rsync mailing list