rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

[Simon Hobson]

Andrew Gideon <c182driver1 at gideon.org> wrote:

> These both bring me to the idea of using some file system auditing 
> mechanism to drive - perhaps with an --include-from or --files-from - 
> what rsync moves.
> 
> Where I get stuck is that I cannot envision how I can provide rsync with 
> a limited list of files to move that doesn't deny the benefit of --link-
> dest: a complete snapshot of the old file system via [hard] links into a 
> prior snapshot for those files that are unchanged.

The think here is that you are into "backup" tools rather than the general purpose tool that rsync is intended to be.

storebackup does some elements of what you talk about in that it keeps a catalogue of existing files in the backup with a hash/checksum for each. I'm not sure how it goes about picking changed files - I suspect it uses "time+size" as a primary filter, but on the other hand I know for a fact you can "touch" a file and that change won't appear in the destination*.
But for remote backups, the primary server can generate a changes list which is then copied to the remote server which then adds the new/changed files and hard-links the unchanged ones according to the list it's been given.
If you turn off the file splitting and compression options, the backup is a series of hard-linked directories which you can look into and pull files directly.
* But if you do alter the timestamp on a file without changing the contents, that will not appear in the file structure in the backup - later "copies" of the file retain the earlier timestamp. It does keep this information, and if you use the corresponding restore tool then you get back the correct timestamp.


In a completely different setup, I also use Retrospect. Recent versions have an option (Instant Scan") to allow the client to keep an audit of changes to avoid the "scan the client/do a massive compare" that's needed with this option turned off.