rsync rewrites all blocks of large files although it uses delta transfer

[Delian Krustev]

On Wednesday, February 13, 2019 6:20:13 PM EET Remi Gauvin via rsync 
<rsync at lists.samba.org> wrote:
> Have you run the nifs-clean before checking this free space comparison?
>  Maybe there is just large amplification created by Rsyn's many small
> writes when using --inplace.

nilfs-clean is being suspended for the time of the backup. It would have idled 
if the fullness threshold of the FS (90% by default) have not been reached.

The problem is probably that these mysqldump files have changed data near the 
beginning of the files. Thus any later blocks have to be overwritten. In order 
to avoid this "rsync" would have to allocate and deallocate space in the 
middle of the file:

  http://man7.org/linux/man-pages/man2/fallocate.2.html

and unfortunately the respective syscalls are not portable, quite new and 
filesystem specific.

Would have been nice to have these for all OSes and filesystems though. And 
better yet not aligned on FS block size. E.g.:

  - give me 5 new blocks in the middle of file F starting at POS
  - do not use the entire last block of these 5 but rather only X bytes of it.

or
  - replace block 5 with "this" partial block data
  - truncate blocks 6 to 20

I can find a usage for them in many application workflows - from text editors 
trough databases to backup software ..


Cheers
--
Delian