Trouble with "--delete" with "--files-from"

[Richard Hopkins]

Hi Wayne...

--On Thursday, July 29, 2004 10:14 AM -0700 Wayne Davison 
<wayned at samba.org> wrote:

> On Thu, Jul 29, 2004 at 03:09:06PM +0100, Richard Hopkins wrote:
>> This is working well for me, except for the "--delete" bit. (When
>> mailboxes  are deleted on the source system I obviously want this
>> reflected on the  target.)
>
> Delete only affects directories that rsync has sent the contents of,
> otherwise it wouldn't know what to delete.  It sounds like you should be
> using --exclude instead of --files-from, like the following (I made a
> guess that the /temp2 arg from your original command was a typo since it
> isn't leagal syntax to have two sources with --files-from).
>
> Put this in a file named "ex":
> + cyrus.seen
> + cyrus.cache
> + cyrus.index
> + */
> - *
>
> ... and run this:
>
> rsync -quWae /usr/local/bin/ssh --delete --exclude-from=ex
>      /mnt2/sms/mail/ user at remote:/var/cyrus/backups/spo-ms
>
> I think the trailing slash is what you want on the source -- only
> remove it if you want the "mail" subdir to be included in "spo-ms".
>

Thanks for this. I suspect that the problem with doing it this way, though, 
is the time it's going to take trawling through the file system. The 
smaller of our two main message stores has over 28,000 mailboxes 
(=directories) and just under 4 million messages (=files).

Using the --files-from approach (by the way, what appeared to be two of 
these was caused by  mis-cutting and pasting - it should have appeared as 
--files-from=tmp/temp2) I can do an initial rsync in just under 5 hours and 
incrementals in about 30 minutes. I suspect that the time it'll take using 
any other approach just won't be feasible. As I write, I can see that the 
rsync job I kicked off 40 minutes ago (using the suggested approach) is 
only at...

lstat64("a/user/am1550/3542.", 0xFFBEC5C8)      = 0
lstat64("a/user/am1550/3543.", 0xFFBEC5C8)      = 0
lstat64("a/user/am1550/3544.", 0xFFBEC5C8)      = 0
lstat64("a/user/am1550/3547.", 0xFFBEC5C8)      = 0
lstat64("a/user/am1550/3548.", 0xFFBEC5C8)      = 0

(am1550 is the 1,424th directory of the 28,117 total)

Perhaps I'll need to adopt a different approach. One that comes to mind is 
that I stick with what I've been using and then maybe do something on the 
backup system to get rid of the directories which I can't "--delete".

Cheers,

Richard