Use rsync's checksums to deduplicate across backups

Dan Stromberg drsalists at
Sat Nov 19 19:11:53 MST 2011

On Sat, Nov 19, 2011 at 5:43 AM, Andrea Gelmini <andrea.gelmini at>wrote:

> 2011/11/3 Alex Waite <alexqw85 at>:
> >    Recently I learned that rsync does a checksum of every file
> > transferred.  I thought it might be interesting to record the path and
> > checksum of each file in a table.  On future backups, the checksum of
> I guess you can be interested in these projects:
> - lessfs:¹ deduplication and compression via fuse (it's a mature project);

This sounds like the garbage collector I wrote when I was kid.  It's a
pretty complex way to do things.  It also doesn't compress very hard - it
kind of has to, because it's intended to be a filesystem.

> - Bup:² it uses git fs to store backup (young but very powerfull).

Does it suffer from the usual backup-to-git problem: No expiration?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the rsync mailing list