file corruption

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rsync by default checksums the whole file from the source vs the whole
file on the target even if the whole file on the target is a
combination of stuff that was already on the target and some stuff
that had to be transferred.

The only time I have ever seen this fail was when the target system
had bad RAM modules that were being used as disk cache.  The system
would verify that it transferred the correct data and then it would
write slightly different data to the disk.  This is why I now insist
on ECC RAM.

The only times I have ever seen rsync's --checksum be useful is when
you want to make sure that rsync does absolutely nothing to unchanged
files (--write-batch) or when you know that hardware has corrupted
your data and you want to force the correct data onto an existing
- --link-dest based backup system.

Outside of that, --checksum is almost always slower than
- --ignore-times (aka re-copy everything) which is slower than simply
- --archive.

Simply put, if you have something that is screwing with your file data
without touching your time stamps then you have been infected with a
rootkit.  You should be thanking rsync for not backing up your
infected files.  Incinerate your infected disk and then restore from
backup.  Since your infection made itself immune from mtime changes it
also made itself immune to being backed up.  Good riddance.

On 04/29/13 00:28, anuragmnnit2004 wrote:
> If full file checksum is matched with source before writing into
> disc. Does it point to disc corruption?
> 
> What rsync options can be used to debug this issue ? Do I need to
> read the destination file again and match the checksum with 
> source.
> 
> 
> Kevin Korb wrote -a should take care of making sure rsync doesn't
> re-transfer things that don't need it.  -c will make sure rsync
> takes longer than just re-copying everything would.
> 
> Rsync does do a checksum of the files it transfers without -c.
> 
> On 04/28/13 03:40, anuragmnnit2004 wrote:
>>>> Database lock is taken on active before rsync is done so it
>>>> is in freezed state.
>>>> 
>>>> I have added checksum so that later full syncing is not done
>>>> on further rsync. Does cheksum is verified before writing
>>>> data to disc in destination?
>>>> 
>>>> 
>>>> 
>>>> Kevin Korb wrote In use databases can't be copied with rsync
>>>> or any file based tool. The database must be inactive to be
>>>> copied.  You didn't mention which kind of database you are
>>>> using but most have a function to freeze the database for
>>>> backups.  I don't know if QNX has LVM functionality but that
>>>> can work too.
>>>> 
>>>> Also, don't use -c unless you have a really good reason.  It
>>>> is almost always a bad idea.
>>>> 
>>>> On 04/27/13 14:26, anurag trivedi wrote:
>>>>>>> I am using rsync in my project for copying database
>>>>>>> binary files over the network from active module to
>>>>>>> standby module. Once in a while files found to be
>>>>>>> corrupt on standby module when copying is done first
>>>>>>> time.When I checked the active module at that time
>>>>>>> files found to be proper. rsync is completing
>>>>>>> successful without giving any errors. On comparing both
>>>>>>> files I see a big hole in standby file filled with
>>>>>>> 0's. Below is my rsync command -
>>>>>>> 
>>>>>>> rsync -a  -c  -v --delete  --timeout=250
>>>>>>> --contimeout=30 --temp-dir=xxxx srcfile    dstfile
>>>>>>> rsync is running in daemon mode. OS - QNX rsync version
>>>>>>> - 3.0.9
>>>>>>> 
>>>>>>> On trying to sync same files again sync happened
>>>>>>> properly.
>>>>>>> 
>>>>>>> Does anybody have similar kind of issues?
>>>>>>> 
>>>>>>> Please help me in this regard.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>> 
>>>>> -- Please use reply-all for most replies to avoid omitting
>>>>> the mailing list. To unsubscribe or change options: 
>>>>> https://lists.samba.org/mailman/listinfo/rsync Before
>>>>> posting, read:
>>>>> http://www.catb.org/~esr/faqs/smart-questions.html
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> -- View this message in context: 
>>>> http://samba.2283325.n4.nabble.com/file-corruption-tp4647327p4647336.html
>>>>
>>>>
>
>>>> 
Sent from the Samba - rsync mailing list archive at Nabble.com.
>>>> 
> 
>> -- Please use reply-all for most replies to avoid omitting the
>> mailing list. To unsubscribe or change options: 
>> https://lists.samba.org/mailman/listinfo/rsync Before posting,
>> read: http://www.catb.org/~esr/faqs/smart-questions.html
> 
> 
> 
> 
> 
> -- View this message in context:
> http://samba.2283325.n4.nabble.com/file-corruption-tp4647327p4647359.html
>
> 
Sent from the Samba - rsync mailing list archive at Nabble.com.
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlF9+RQACgkQVKC1jlbQAQfRAwCg4VouqtafFbNPkx2eLLYFf3Ss
MNwAoJ4N1ikpEM+v95JOSNwMciowvXjK
=Mi02
-----END PGP SIGNATURE-----