How does rsync file verification work?

[Henri Shustak]

There are a number of data verification tools which may be options. One possibility is FingerPrint, a tool designed to verify data sets (not designed as a trip wire) : http://bit.ly/fingerprint-oriontransfer

Also another thing to keep in mind is that some file systems / hardware storage systems will perform checksums for data so that in the event that the hardware's ECC check is not able to resolve the problem the file system / hardware will detect the issue and attempt repair.

Finally, keep in mind that if you have a lot of data, this verification process may use considerable resources on your system(s).

Hope this helps.

---------------------------------------------------------------------
This email is protected by LBackup, an open source backup solution.
http://www.lbackup.org


> 
> Rsync does not verify the data after it is written to the disk.  The
> whole file checksum is only done to ensure that the file crossed the
> network intact.
> 
> This is not unusual.  Backup programs generally don't re-read from the
> target to confirm a successful write.  In fact if they did they would
> probably just be reading from the cache anyway.
> 
> If you are really paranoid about this then you need a separate file
> verification tool.  Plus you need to make sure the cache is dumped
> between the rsync run and the file verification run.
> 
> Note that the only time I have seen rsync write corrupt files was when
> a bad DIMM was corrupting data in the cache.  The solution to that is
> ECC RAM.  The only other way I have heard of it happening is a failing
> disk in the backup system and the solution to that is SMART monitoring.
>> 
>> I want to use rsync to create backups. I think I need a
>> verification that: 1) data is written to disk. 2) data is read back
>> from disk. 3) verify the checksum of data. Does the automatic file
>> verification in rsync works like this? Technically, I'd like to
>> know how does we make step 2 read data from disk and not from file
>> cache/buffer in OS? Or, do OS/harddrives always perform written
>> data verification anyway so step 2 is not really needed?