Feature request: Store diffed files separately.

[Matt McCutchen]

On Tue, 2008-04-01 at 16:12 -0700, Ben Wilber wrote:
> I would like to be able to
> store diffed files in a separate directory on the receiver so it makes
> doing an incremental backup easy and efficient.  Let me explain:

For completeness, I'll point out that you can accomplish what you
originally wanted with --compare-dest, although I think the --link-dest
approach Brock mentioned is better.

> Day 1: take a full backup of a directory: rsync -r ./source/ ./dest/
> Day 2: Diff the contents of ./source/ and ./dest/ and store only the
> new/modified files in ./incremental_day1

rsync -r --compare-dest=../dest/ ./source/ ./incremental_day1/

> Day 3: Diff the contents of ./source/ and ./dest/ and store only the
> new/modified files in ./incremental_day2

I take you to mean that incremental_day2 contains files created/modified
since the original dest, not since incremental_day1.  The command is
similar:

rsync -r --compare-dest=../dest/ ./source/ ./incremental_day2/

If you wanted to create "chained" incremental backups, the command is:

rsync -r --compare-dest=../incremental_day1/ \
    --compare-dest=../dest/ ./source/ ./incremental_day2/

Unfortunately, the semantics of --compare-dest are not quite right: this
command will omit files that changed back to their original state on day
2, but those files are needed to restore day 2 correctly.

> This way to restore a backup of Day 2 just do: rsync -Pvr
> ./incremental_day2 ./dest/ # But make the updates to ./source/ instead
> so you can preserve existing backups.

The restore command would be:

rsync -Pvr ./incremental_day2/ ./dest/ ./source/

When you give multiple source arguments, files from earlier source
arguments take priority over corresponding files from later source
arguments.  Thus, the command will restore the incremental_day2 version
of a modified file, not the dest version.

Files that are present in dest but *deleted* in incremental_day2 will be
restored, which is wrong.  To avoid this, you could save a "find ."
listing of the source in each incremental backup and restore only the
files in this listing using --files-from.

Matt