using "--checksum" for local sync

[ff0]

> I could just have synced the file anyway?

First important note to make known up front:
When rsync qualifies a file for transfer, then by default it performs in-flight transfer checksums to verify the SRC and DST are binary same.

The --checksum option is an extra step in determining "Does this file need to be updated?" and the sums are calculated on the SRC and DST first before any transfer starts.
So --checksum can be advantageous to mitigate unnecessary (binary same) transfers over slower/metered WAN links.

rsync by default checks if each file's size and time of last modification match between the sender and receiver.
by default if there is no difference, no transfer.

if there is a difference then the transfer takes place.
1x read of the SRC bytes, 1x write of the DST bytes.

This is very efficient BUT there is no data integrity guarantee that SRC and DST are binary same since the last transfer.

Using --checksum mode, the default behaviour changes if the file sizes are equal:
1x read of the SRC bytes for checksum, 1x read of the DST bytes for checksum, if different, 1x read of the SRC bytes for transfer, 1x write of the DST bytes.

So --checksum will add overhead (for files that match in size), for small files its likely unnoticeable and provides an extra guarantee that the source and destination are binary same.

Consider this extract from the man:

rsync man > --checksum option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), so this can slow things down significantly.

I do suggest reading more in the --checksum man section, its very informative.

If a data integrity guarantee is mission critical for your use case(s), then --checksum is awesome for that.

I suppose *cheap* way to force a transfer (skipping --checksum) is to "touch" the file(s) on the SRC that you want to transfer.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Friday, August 27th, 2021 at 11:38 AM, Fourhundred Thecat via rsync <rsync at lists.samba.org> wrote:

> Hello,
>
> I am using the "--checksum" option, when syncing files over the network.
>
> But when syncing large files locally, does it make sense to use
>
> --checksum, or would it slow things down?
>
> I mean, if it needs to calculate the checksum first, it could just have
>
> synced the file anyway. right?
>
> thanks
>
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Please use reply-all for most replies to avoid omitting the mailing list.
>
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html