rsync --delete problem

[Smemoe, Richard L.]

Wayne,

Thanks for your reply.  I upgraded to 2.6.8 and switched to
--delete-during.  But I still have the same problem as I did with
--delete.  I'll try to explain the behavior I'm seeing a little better.
The directory I'm trying to rsync from has ~3.5 million files with the
upper level directory having about 2000 directories.  I'm running 'cd
/backup/directory', then 'rsync -a --delete-during
/backup/directory/directorytocopy .' on every directory in this top
level directory.  If I leave off the --delete-during (or --delete),
everything works fine (except files removed from the original filesystem
aren't deleted), but if I use the --delete-during option, I only end up
with a small fraction of the original filesystem in my backup.  It looks
like it's only copying the changes to the original filesystem since the
last rsync.  I'm not 100% sure that this is what's happening though.
The original filesystem has directories starting with a* to z*, but the
resultant backup filesystem only has directories from m* to z*, so maybe
it is deleting directories from earlier in the filesystem...I'm not
sure.  I have tried changing options in my backup script to get
different behavior, but roughly the same thing happens every night when
I run the script.  How would you recommend running the rsync copy?

Also, after upgrading to 2.6.8, I tried copying the whole filesystem at
once with 'cd /backup/directory; rsync -a --delete-during
/production/filesystem/ .', but this gave me nothing in my backup
filesystem.

Thanks,

Richard

-----Original Message-----
From: Wayne Davison [mailto:wayned at samba.org] 
Sent: Thursday, June 01, 2006 11:37 AM
To: Smemoe, Richard L.
Cc: rsync at lists.samba.org
Subject: Re: rsync --delete problem

On Thu, Jun 01, 2006 at 09:31:49AM -0500, Smemoe, Richard L. wrote:
> When I added the -delete option, my backup started taking about two
> days to finish.  

You'd be better off using --del (--delete-during) than --delete, as
it doesn't require a separate scan through the destination hierarchy.
Also, older versions of rsync created an additional in-memory
representation of the entire destination file tree to do the delete,
so it is possible that such an older version caused rsync to start
swapping, which could easily slow it down by an order of magnitude.
So, make sure you're using a modern version (at least 2.6.4).

As for any incorrect behavior of --delete, you'll need to be more
explicit in what you think it is doing wrong.

..wayne..