rsync doesn't checksum for local transfers?

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I also want to add again that if that additional rsync --checksum run
doesn't use --itemize-changes you won't know if the file is different
for some other reason.  Here is an example of what I mean....

% rsync -vai /etc/hosts /tmp/	#initial file
sending incremental file list
> f+++++++++ hosts

sent 1,920 bytes  received 35 bytes  3,910.00 bytes/sec
total size is 1,835  speedup is 0.94
% rsync -vainc /etc/hosts /tmp/
sending incremental file list
#	no difference found
sent 54 bytes  received 12 bytes  132.00 bytes/sec
total size is 1,835  speedup is 27.80 (DRY RUN)

# edit file and change 1 character without altering size

% rsync -vainc /etc/hosts /tmp/
sending incremental file list
> fc.t...... hosts	# file has a different timestamp. different
> content
expected

sent 61 bytes  received 19 bytes  160.00 bytes/sec
total size is 1,835  speedup is 22.94 (DRY RUN)
% touch --reference=/etc/hosts /tmp/hosts	# force correct mtime
% rsync -vainc /etc/hosts /tmp/
sending incremental file list
> fc........ hosts	# something is wrong here!

sent 57 bytes  received 19 bytes  152.00 bytes/sec
total size is 1,835  speedup is 24.14 (DRY RUN)


On 12/04/2014 07:17 PM, Shriramana Sharma wrote:
> On Fri, Dec 5, 2014 at 4:30 AM,  <devzero at web.de> wrote:
>> After being written to disk, for both local and remote transfers,
>> the destination file as a whole is not being re-read for
>> checksumming. Checksumming is only being done for the
>> reconstruction process: The checksum is calculated across the
>> bits being received and the bits being read from the target file,
>> so essentially the updated target file is being checksummed while
>> it`s being written to.
> 
> Some slight modifications, including clarifications from Kevin:
> 
> For remote transfers, the integrity of the data is verified by 
> checksumming on-the-fly at the sending/receiving rsync instances.
> For local transfers, the reading and writing are done by the same
> rsync instance so such integrity verification is not required
> unless you have bad RAM or storage. This means that for both local
> and remote transfers, the destination file as a whole is not being
> re-read for checksumming.
> 
> Note that sync may be required at the destination to actually
> ensure that the received files are written to storage. Also note
> that an extra rsync -c with the same source/target arguments as the
> first time may be done to ensure that the received/stored files are
> checksum-wise identical to the source by re-reading the files.
> Again, this is only useful if the kernel will actually be accessing
> the data from disk such as after clearing its read-write cache.
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlSBBJQACgkQVKC1jlbQAQeYHACg24vsMgAXipxy0NOgN9+vhkTn
N7gAnA8nb6RT4fwbytje/ivw9yu17z7v
=zHJq
-----END PGP SIGNATURE-----