rsync / checksum small block / xfer small block

[Jamie Lokier]

alexus wrote:
> not quite what i need
> 
> let's take another example
> 
> let's say i have a mysql db, and only few rows gets changed on daily
> basis, yet that data file itself is huge, so rsync checks for checksum
> sees that it's different and xfer the whole file, i use remote site,
> so xfer takes long time and plus we pay for our bw, so it cost us
> money too, instead there is a technology out there called dedupe, it
> will slice file for many blocks ( i think 12k each ) and it will
> checksum each of blocks, and will only trasnfer the one that were
> changed, this way, if very little get changed during the day, the xfer
> of large file will dramaticly drop, as it will only xfer one small
> block 12k instead of 1G for instance...
> 
> so i was wondering if rsync ever going support this kind of
> technology, as today there is more and more of data and not enough
> time to back it up...

rsync has always supported this block checksum technology like "dedupe".
It was the original reason for writing rsync!

rsync is smarter than just checksumming every block, though, as it can
detect arbitrary insertions and deletions too.  That is what Matt
means by delta-transfer algorithm.

If rsync is transferring the whole file in your example, there's
something wrong with your command line options or your measurements.

-- Jamie