rsync --delete* Bug or feature-request

[Matt McCutchen]

On Tue, 2008-07-29 at 17:19 +0200, Gabriel CORRE wrote:
> I'm trying to do a differential backup. Each commands/scripts are launch on backup server side.
> 
> - rsync-ref: 1 script perform a complete backup to a directory "ref".
>    something like that:
>      rsync --archives \
>        station:/path1/ /servbasepath/ref/servpath1/
> 
> - rsync-diff: 1 script perform a differential backup to a directory "diff".
>    something like that:
>      rsync --archives \
>        --compare-dest=/servpath1/ \
>        --delete \
>        station:/path1/ /servbasepath/diff/servpath1/
> 
> 
> See the following scenario:
> 
> on serv:
>   - rsync-ref is done
>     Result: large number of files are synchronized.
> on station:
>   - the user create a new file "test"
> on serv:
>   - rsync-diff is done
>     Result: the new file "test" was synchronized in diff directory.
>   - do rsync-ref
>     Result: the new file "test" was synchronized in ref directory.
>   - do rsync-diff
>     Result: nothing done, all file are synchronized.
>     That is my problem:
>     the file "test" in diff directory may/must be deleted,
>     because it is already on the ref directory.
> 
> 
> I think --delete (or some other --delete-* options too) should remove
> the file because it is already found in ref directory.
> 
> But Is it really a bug ?
> or may I request a --delete-if-found-in-compare-dir ?

Rsync is behaving as expected.  The man page description of --link-dest
says, "rsync treats existing files as definitive (so it never looks in
the link-dest dirs when a destination file already exists)"; it would be
good to add that the same is true of --compare-dest.

You are welcome to request an option to make rsync delete destination
files that are matched in --compare-dest dirs.  I would actually suggest
a more general option called --recheck-basis-dirs that also replaces a
destination file matched in a --link-dest dir with a hard link of the
basis file.  (I don't think the analogous behavior for --copy-dest would
be useful.)

> For workaround I run a script that find duplicated copies of files on
> the both directories. And run the rsync-diff after.
> But I work on more than 6 millions of files, the rsync and find-scrtipt took a very lot of time.
> Rsync already compare the both dir with --compare-dest then
> it will probably easily append duplicated files to the delete queue.

Another approach would be to run rsync-diff to a new destination each
time, but I imagine you want to reuse the data in the old destination
instead of retransferring it all over the network.  The command

rsync --compare-dest=/servbasepath/ref/servpath1/ \
	--copy-dest=/servbasepath/diff/servpath1.old/ \
	station:/path1/ /servbaseepath/diff/servpath1/

would do that, except that rsync currently doesn't support mixing
different types of --*-dest options on the same run; see:

http://lists.samba.org/archive/rsync/2007-February/017210.html

Matt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/rsync/attachments/20080729/0b4b83fd/attachment.bin