Verifying backups

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

First off, --fileflags --force-change are not in my man rsync so I
don't know what those are.

Second, you should look into using either ZFS subvolume snapshots or
rsync --link-dest to maintain multiple backups.

Now, for your actual question...
Add --itemize-changes to your standard command line.  -v is almost
entirely useless without it anyway.

To figure out and fix what is corrupt you have 2 paths:

1. Add --checksum for a single pass.  This will take forever as rsync
checksums everything even things it shouldn't expect to match (even
things that are only on one end!).  Anything that checksum finds that
rsync wouldn't have otherwise found would have a 'c' but not a 't'.

2.  Add --ignore-times for a single pass.  Normally this doesn't take
as long as --checksum.  However, since you are using an external USB
device which means you are also using --whole-file this will probably
be even slower than --checksum.

Either way, you need to get your system writing files correctly.

On 09/30/2015 04:19 PM, Ronald F. Guilmette wrote:
> 
> For some time now I've been using rsync on FreeBSD to make my
> system backups.  Recently, I accidentally rm'd some files from one
> directory and I had to go and fetch copies off of my backup drive.
> After I had done so, I found that about 1/5 of them were corrupted.
> (They were all .jpg files, by the way.)
> 
> To be clear, I most certainly *do not* attribute the apparent
> corruption to rsync.  My backup drive is itself slightly dubious...
> a retired (so-called ``refurbished'') WD Black 1TB drive which the
> S.M.A.R.T. info showed already had well more than 30,000 hours on
> it before I ever laid hands on it.  Also, and separately, I've been
> trying to use it... and others like it... in one of these
> open-on-the-top slot loading external/powered SATA to USB3
> adapters.  As I have learned recently, 2.5" drives are generally no
> problem to use with such adapters, but unless you have a utility
> fan pointing right at it, 3.5" drives that are placed into one of
> those adapters can pretty quickly get REALLY hot... a fact which,
> taken alone, may actually be the root cause of the file corruption
> on my backup drive.
> 
> So anyway, I have a 1TB backup drive now that is chock full of 
> files that I placed on it previously (using rsync)... the majority 
> of which, by volume, are binary media files (i.e. mp3 songs,
> JPEGs, movies, etc.) and now I'd like to carefully check them all
> to see which ones may have gotten corrupted.
> 
> So, my question:  How best to do this?
> 
> Looking at the rsync man page, I see a couple of options that may 
> perhaps be relevant, specifically:
> 
> -c, --checksum              skip based on checksum, not mod-time &
> size -n, --dry-run               perform a trial run with no
> changes made
> 
> Can I use these to, in effect, check all of my backup files for
> integrity?
> 
> Please note that the options I have been using to make my backups
> are as follows:
> 
> -v -t -axHAXS --delete --fileflags --force-change
> 
> I don't imagine that either the -n or -c options will interact
> badly with any of those, correct?
> 
> 
> Regards, rfg
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
- -*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
- -*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlYMSlEACgkQVKC1jlbQAQeNCQCff5zt9XGv94L0uNCtZn6Yf1L5
WqEAoOit1g8CazYDIwp9YDjMDPMhU4Tr
=ekEj
-----END PGP SIGNATURE-----