file corruption

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have seen this behavior before.  Twice.

Both times the cause was bad RAM on the target system.  The bad RAM
was corrupting the files within the disk write cache so that rsync
believed it was writing the correct data but the disk was not getting
the correct data.  Ever since that happened to me I have insisted on
ECC RAM and have never had such a problem again.

It is also possible that a disk or disk controller could be at fault.

Rsync does do an in memory checksum before it renames the file into
place but it does not re-read the file back from the disk (which in
reality would also require a dump of the cache).

On 03/08/13 20:55, xiaolong mou wrote:
> Hi,
> 
> I am backing up about 500G of data from a linux-based NAS to a USB 
> hard drive attached to a router (rt-n16 with tomatousb firmware).
> Both NAS and the router have rsync-3.0.9. The router is running
> rsync in the daemon mode. To test the set up, I rsync'd the files
> to the empty USB hard drive, then back to the NAS in a new temp
> dir. Afterwards, I did a local "diff -r" on the NAS. To my
> surprise, a few files were corrupted (only 3 out of 17K files).
> "cmp -l" shows single byte difference between the original and
> rsync'd files. However, I didn't see any error message on the NAS
> side or in the rsyncd logs. How is that possible? Doesn't rsync
> always do a checksum verification after copying the files?  I have
> been using rsync for years for local backup. The feeling of silent
> file corruption is scary. Could someone point me to the right
> direction? I really want to get to the bottom of this. Much
> appreciated.
> 
> Regards, Xiaolong
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlE6mckACgkQVKC1jlbQAQdVNQCfYrAIAy3flzK3VK0/mrz7W3y0
byUAn3gHDtDWgcg9rZODPKOyCImd+vuk
=DzDh
-----END PGP SIGNATURE-----