Using reflink copy/FICLONE when updating files

Christoph Biedl cbiedl at gmx.de
Thu May 20 15:33:29 UTC 2021


Hello,

at first, I'm a bit surprised this hasn't been requested before - or did
I miss a bit?

When updating an existing file, the receiving rsync (at least in
general) creates a copy of the existing file and does all writing there
before eventually discarding the old file. This approach is fine for
data integrity and atomic replacement but creates a lot of I/O and
consumes disk space. This becomes a problem when updating containers of
several Gigabytes in size and - like in my case - resuming an
interrupted transfer done using --partial.

However, that situation could be avoided if rsync would reflink-copy the
existing file first, quite like cp --reflink=auto in the coreutils. If
that fails since the underlying file system does not support it, rsync
would fall back to the existing code.

Is that doable?

Honestly, I didn't dare to implement a prototype. Possibly it was
sufficient to add some code to receive_data in receiver.c, in the block
that follows "if (append_mode > 0) {". Actions would be mostly
"ioctl(fd, FICLONE, fd_r);", and do the checksuming. So it seems the I/
part is still necessary but at least /O can be avoided. But I might be
completely wrong here.

Kind regards,

    Christoph



More information about the rsync mailing list