superlifter design notes and a new proposal

Donovan Baarda abo at
Tue Aug 6 05:59:02 EST 2002

On Tue, Aug 06, 2002 at 02:45:47PM +1000, Martin Pool wrote:
> I was talking to Tim the other day about the backup vs transfer
> dichotomy identified by John.
> One interesting idea, further down the track, would be to introduce a
> virtual filesystem layer into rsync 3, similar to that in Samba, so
> that all disk IO goes through a function-table layer.
> You could then write a filesystem layer that, rather than talking to
> the native filesystem, talks to some kind of database.  This would be
> very nice for a backup server for a couple of reasons.  (It could be a
> Sleepycat DB or SQL or whatever.)
> Firstly, the database could implement arbitrarily rich filesystem
> semantics without being limited by the native filesystem.  For
> example, you couldn't back up OS X resource forks to a Solaris
> machine, but you can easily store them inside a database.  Similarly
> for storing extended attributes, or storing NT security descriptors on
> Unix (or vice versa).
> Secondly, the database could be tuned for the particular case of
> storing incremental backups: between one version of a file and the
> next, you could just store an xdelta diff, and identical files could
> be replaced by something similar to hardlinks.  The whole thing could
> be compressed.  Basically you would be tuning for the case of an
> append-only filesystem.

Hmmm, sounds like you are re-inventing xdelta2... uses a bdb backend, has
support for extended attributes, and even has an NFS interface...

If you are considering this, I thoroughly recommend looking at xdelta2
first. Probably worth speaking to jmacd too, because he has already been
there and will probably have comments about ways he could make it better.

ABO: finger abo at for more info, including pgp key

More information about the rsync mailing list