rsync rewrites all blocks of large files although it uses delta transfer

[Delian Krustev]

  Hi All,

For a backup purpose I'm trying to transfer only the changed blocks of
large files. Thus I've run "rsync" with the appropriate options:

	RSYNC_BKPDIR=`mktemp -d`
	rsync \
		--archive \
		--no-whole-file \
		--inplace \
		--backup \
		--backup-dir="$RSYNC_BKPDIR" \
		--verbose \
		--stats \
		/var/backups/mysql-dbs/. \
		/mnt/bkp/var/backups/mysql-dbs/.

The problem is that although "rsync" shows that delta transfer is used(when 
run with -vv) and only small amount if data is transferred, the target files 
look to be overwritten in full.

Here is the output of "rsync" and some more debugging info:


####################################################
sending incremental file list
./
horde.data.sql
horde.schema.sql
LARGEDB.data.sql
LARGEDB.schema.sql
mysql.data.sql
mysql.schema.sql
phpmyadmin.data.sql
phpmyadmin.schema.sql

Number of files: 9 (reg: 8, dir: 1)
Number of created files: 0
Number of deleted files: 0
Number of regular files transferred: 8
Total file size: 1,944,522,704 bytes
Total transferred file size: 1,944,522,704 bytes
Literal data: 21,421,681 bytes
Matched data: 1,923,101,023 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 21,612,218
Total bytes received: 323,302

sent 21,612,218 bytes  received 323,302 bytes  259,591.95 bytes/sec
total size is 1,944,522,704  speedup is 88.65

# du -m 1.9G /tmp/tmp.8gBzjNQOQZ
1.9G /tmp/tmp.8gBzjNQOQZ

# tree -a /tmp/tmp.8gBzjNQOQZ
/tmp/tmp.8gBzjNQOQZ
├── horde.data.sql
├── horde.schema.sql
├── LARGEDB.data.sql
├── LARGEDB.schema.sql
├── mysql.data.sql
├── mysql.schema.sql
├── phpmyadmin.data.sql
└── phpmyadmin.schema.sql

0 directories, 8 files

Free space at the beginning and end of the backup:
Filesystem             1M-blocks   Used Available Use% Mounted on
/dev/mapper/bkp           102392  76872     20400  80% /mnt/bkp
/dev/mapper/bkp           102392  78768     18504  81% /mnt/bkp

####################################################

As can be seen "rsync" has sent about 20M and received 300K of data. However 
the filesystem has allocated almost 2G, which is the total size of the files 
being backed up.

The filesystem mounted on "/mnt/bkp" is of type "nilfs2", which is a log 
structured filesystem. I'm using its snapshotting feature to keep backups for 
past dates.


Is there anything that can be done in order "rsync" to overwrite only the 
changed blocks ?




P.S. I guess that it will be the same for copy-on-write filesystems, e.g. 
BTRFS or ZFS.



Cheers
--
Delian