single rsync of large directory structure versus several
jw at pegasys.ws
Sun Mar 16 20:40:54 EST 2003
On Sun, Mar 16, 2003 at 09:05:45AM +0000, Pete Mocat wrote:
> I am working on a backup script using rsync and have a question.
> Rather than doing a single rsync of /home/ I have set my script up
> to loop over /etc/passwd and do a seperate rsync of each user home dir
> (/home/user1, /home/user2) etc.
> In the end, I should have the exact same backup either way. However I feel
> safest doing it as multiple rsyncs (one for each user). I am doing this
> because I want to avoid a case in which some fatal error on a particular
> dir kills off the whole rsync. For example, given:
> If I do:
> rsync -vaze ssh /home/
> I am afraid that if /home/adoe had a fatel error (one that kills rsync) all
> dirs past it would not be backed up and given an over-worked sysadmin
> (never!) we might miss the error until the day we need the backups.
> If I do a seperate:
> rsync -vaze ssh /home/adoe
> rsync -vaze ssh /home/jdoe
> rsync -vaze ssh /home/zdoe
> I should still have a backup of each dir even if some fail.
> Is this being too paranoid?
How paranoid is too paranoid?
> Is there something built into rsync to avoid this?
If an error looks to be isolated to a single file rsync
continues onward and merely logs the error. Your script
should examine the exit status and/or logs and notify if
there is an error.
> Does anyone have any better ideas for me? I want to make the best use of
> rsync possible.
> Thanks for your advice and comments!
I don't think you are being paranoid enough. Unless your
backups are subsequently backed up again you only have
one backup here. What is going to happen if a file gets
trashed and it isn't noticed for a few days?
I would suggest you look into Mike Rubel's snapshot
system, Dirvish or another system that has already dealt
with these issues and can maintain and manage multiple
backup images. I cannot speak for the others but i know
that Dirvish is in production use at multiple sites.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync