Confused as to why rsync thinks time, owner and group of many files differ

[Andy Smith]

Hi,

I am at the moment using rsync to move quite a big set of backups
from one machine to another. The source filesystem is xfs; the
target filesystem is btrfs.

For various reasons I have been stopping the rsync part way through
and re-starting. I have noticed that a large number of files are
transferred over and over and I can't work out why.

Example:

sudo rsync -iPva \
    --inplace \
    --numeric-ids \
    --delete \
    /data/backup/rsnapshot/daily.0/cacti/ \
    root at koff:/data/backup/rsnapshot/daily.0/cacti/

...
<f..t.og... var/www/index.html
          5,258 100%    5.78kB/s    0:00:00 (xfr#1276, to-chk=1/43437)

If I run the rsync command again, thousands of lines of output will
appear again, all showing itemized changes for 't' and sometimes
'p', 'o' and 'g'. Notably, var/www/index.html will keep appearing in
the list.

Let's have a look at that file.

Source:

$ stat /data/backup/rsnapshot/daily.0/cacti/var/www/index.html
  File: /data/backup/rsnapshot/daily.0/cacti/var/www/index.html
  Size: 5258            Blocks: 16         IO Block: 4096   regular file
Device: fd05h/64773d    Inode: 354337      Links: 37
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2022-02-03 04:53:12.115719681 +0000
Modify: 2006-07-14 16:42:37.000000000 +0000
Change: 2022-01-01 17:31:28.553758359 +0000
 Birth: -

Destination:

$ stat /data/backup/rsnapshot/daily.0/cacti/var/www/index.html
  File: /data/backup/rsnapshot/daily.0/cacti/var/www/index.html
  Size: 5258            Blocks: 16         IO Block: 4096   regular file
Device: 26h/38d Inode: 7534065     Links: 26
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2022-01-25 20:40:09.930960486 +0000
Modify: 2006-07-14 16:42:37.000000000 +0000
Change: 2022-02-03 21:45:44.194559899 +0000
 Birth: 2022-01-25 20:40:09.930960486 +0000

When rsync considers times as being different, it means mtime,
right? Yet these files have identical mtimes. They also have
identical uid, gid and permissions.

I would not expect this and other files like it to keep being
listed for change over and over again. I can tell by the summary
that the actual contents of the files weren't sent, so at least it
didn't try to send all the data again. But even if rsync did
consider these files to have different mtime/uid/gid, should it not
have written that and be happy at next run?

rsync versions:

Source:

$ rsync --version
rsync  version 3.1.2  protocol version 31
Copyright (C) 1996-2015 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

Destination:

$ rsync --version
rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    SIMD, asm, openssl-crypto
Checksum list:
    xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

What am I missing?

Thanks,
Andy