help troubleshooting inconsistencies in back up sizes

[John Van Essen]

On Thu, 3 Feb 2005, gaw zay <gauze at dropdead.org> wrote:
....
> I use this shell script to back up:
> 
> for i in a b c d e f g h i j k l m n o p q r s t u v w x y z `seq 0 9`; do
>         /usr/local/bin/rsync -a -z -W --delete /mailhome/$i/ user at backup:/mailhome/$i
> done
>
> question one would be does this look correct?

Your script looks OK, but it helps to use at least one -v option
so you can see what files are being transferred.  If you have a busy
mailserver at the time of the rsync, you might be needlessly
transferring lock files and temporary files as mailboxes are updated.

--stats would help you see the size of the hierarchies, too.

You can remove -v and --stats once you get an idea of the activity.

> now here is the heart of the problem:
> on server1 the partition is 121G
> on server2 it's 110G
> on server3 it's 118G

When you say the partition is NNN, do you mean that's the used space
as reported by df?

> so I assume I have multiple problems here. I don't see --progress as being
> usable in my case since I have such a large amount of files, how can I
> debug what the differences are between these 3 body of files that doesn't
> involve actually checking them individually? I basically want to be
> informed of errors of any kind (io, permissions) as my logging of
> stderr/stdout doesn't show anything, which makes me think many the problem
> is my command line but it sure looks ok to me. I'm using version 2.6.3 of
> rsync btw.

If you don't have -v or --stats or --progress then there will be
no output for a successful rsync.

You should be getting any error messages that may be generated.

You could check the exit code from rsync and print your own message
if it is non-zero to ensure that you get a message when there's
an error.

To examine the difference between hierarchies, do this on each server
(replacing N with a number):

  find /mailhome -ls | sort +10 >/tmp/serverN.txt

(The sort sorts on the filename at the end of the line.)

If the machines use different timezones, put TZ=GMT before the find:

  TZ=GMT find /mailhome -ls | sort +10 >/tmp/serverN.txt

so all the timestamps will be the same.

Then use rsync to gather them on one machine and do diffs between them.
-- 
        John Van Essen  Univ of Minn. Alumnus  <vanes002 at umn.edu>