inefficient: --checksum calculation shouldn't be done for new files
Carlos Carvalho
carlos at fisica.ufpr.br
Mon Jul 11 18:18:52 MDT 2011
Wayne Davison (wayned at samba.org) wrote on 4 July 2011 17:10:
>On Sat, Jul 2, 2011 at 5:46 PM, Carlos Carvalho <carlos at fisica.ufpr.br> wrote:
>
> When --checksum is used they're calculated in both ends to see if the file
> should be transfered. This is of course not necessary if the file doesn't
> exist in the destination. However, the checksum is still calculated by the
> sender, which is often a very large overhead.
>
> Would it be possible to avoid it?
>
>
>To do so would involve adding an extra round-trip request to a transfer, so it
>is feasible, but is not currently supported.
[...]
>That all sounds interesting, but would require a new
>--favor-missing-files (or some such) option to tell rsync to use the
>alternate checksum method. It would be interesting to try something
>like that and see how much time it saves in checksum generating vs
>time it consumes in round-trip lag.
Understood. I asked just in case it was an easy optimization but it
looks like some significant complication for a rather rare use.
>As for what is currently possible, see the patches/db.diff, patches/
>checksum-reading.diff, patches/checksum-updating.diff, and
>(possibly) patches/ checksum-xattrs.diff patches for example ways to
>make the checksum sending more efficient.
I can't use this because we're the destination and the origins are
spread all over the world. Instead I've separated the files we need to
checksum and do them in a different rsync run.
Thanks for the detailed explanation.
More information about the rsync
mailing list