single rsync of large directory structure versus several smaller rsync's

[jw schultz]

On Sun, Mar 16, 2003 at 09:05:45AM +0000, Pete Mocat wrote:
> Hello,
> 
> I am working on a backup script using rsync and have a question.
> 
> Rather than doing a single rsync of /home/ I have set my script up
> to loop over /etc/passwd and do a seperate rsync of each user home dir 
> (/home/user1, /home/user2) etc.
> 
> In the end, I should have the exact same backup either way. However I feel 
> safest doing it as multiple rsyncs (one for each user). I am doing this 
> because I want to avoid a case in which some fatal error on a particular 
> dir kills off the whole rsync. For example, given:
> 
> /home/adoe
> /home/jdoe
> /home/zdoe
> 
> If I do:
> 
> rsync -vaze ssh /home/
> 
> I am afraid that if /home/adoe had a fatel error (one that kills rsync) all 
> dirs past it would not be backed up and given an over-worked sysadmin 
> (never!) we might miss the error until the day we need the backups.
> 
> If I do a seperate:
> 
> rsync -vaze ssh /home/adoe
> rsync -vaze ssh /home/jdoe
> rsync -vaze ssh /home/zdoe
> 
> etc.
> 
> I should still have a backup of each dir even if some fail.
> 
> Is this being too paranoid?

How paranoid is too paranoid?

> Is there something built into rsync to avoid this?

If an error looks to be isolated to a single file rsync
continues onward and merely logs the error.  Your script
should examine the exit status and/or logs and notify if
there is an error.

> Does anyone have any better ideas for me? I want to make the best use of 
> rsync possible.
> 
> Thanks for your advice and comments!

I don't think you are being paranoid enough.  Unless your
backups are subsequently backed up again you only have
one backup here.  What is going to happen if a file gets
trashed and it isn't noticed for a few days?

I would suggest you look into Mike Rubel's snapshot
system, Dirvish or another system that has already dealt
with these issues and can maintain and manage multiple
backup images.  I cannot speak for the others but i know
that Dirvish is in production use at multiple sites.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt