inefficient: --checksum calculation shouldn't be done for new files

Carlos Carvalho carlos at fisica.ufpr.br
Mon Jul 11 18:18:52 MDT 2011


Wayne Davison (wayned at samba.org) wrote on 4 July 2011 17:10:
 >On Sat, Jul 2, 2011 at 5:46 PM, Carlos Carvalho <carlos at fisica.ufpr.br> wrote:
 >
 >    When --checksum is used they're calculated in both ends to see if the file
 >    should be transfered. This is of course not necessary if the file doesn't
 >    exist in the destination. However, the checksum is still calculated by the
 >    sender, which is often a very large overhead.
 >
 >    Would it be possible to avoid it?
 >
 >
 >To do so would involve adding an extra round-trip request to a transfer, so it
 >is feasible, but is not currently supported.
[...]
 >That all sounds interesting, but would require a new
 >--favor-missing-files (or some such) option to tell rsync to use the
 >alternate checksum method.  It would be interesting to try something
 >like that and see how much time it saves in checksum generating vs
 >time it consumes in round-trip lag.

Understood. I asked just in case it was an easy optimization but it
looks like some significant complication for a rather rare use.

 >As for what is currently possible, see the patches/db.diff, patches/
 >checksum-reading.diff, patches/checksum-updating.diff, and
 >(possibly) patches/ checksum-xattrs.diff patches for example ways to
 >make the checksum sending more efficient.

I can't use this because we're the destination and the origins are
spread all over the world. Instead I've separated the files we need to
checksum and do them in a different rsync run.

Thanks for the detailed explanation.


More information about the rsync mailing list