rsync --link-dest and --files-from lead by a "change list" from some file system audit tool (Was: Re: cut-off time for rsync ?)

Simon Hobson linux at thehobsons.co.uk
Tue Jul 14 13:55:41 UTC 2015


Ken Chase <rsync-list-m829 at sizone.org> wrote:

> And what's performance like? I've heard lots of COW systems performance
> drops through the floor when there's many snapshots.

For BTRFS I'd suspect the performance penalty to be fairly small. Snapshots can be done in different ways, and the way BTRFS and (I think) ZFS do it is actually quite elegant.

Some systems keep a "current" state, and separate "files" for the snapshots (effectively a list of the differences from the current version). The performance hit comes when you update the current state, but before writing a chunk, the previous current version of the chunk must be read and added to the snapshot(s) that include it.

I believe the way BTRFS and XFS do it is far more elegant. When you write a file out, you stuff the data in a number of disk blocks, and write an entry into the filesystem structures to say where that data is stored.
In BTRFS, when you do a snapshot, it just "notes" that you've done it and at that point very little happens.
When you then modify a file, instead of writing the data to the same blocks on disk, it's written to empty space, the old version is left in place, and the filesystem structures are updated to account for there now being two versions. If you only write some blocks of the file, I'd assume that only those new blocks would get the COW treatment.
So the only overhead is in allocating new space to the file, and keeping two versions of the file allocation map.
When you delete a snapshot, all it does is delete the snapshotted versions of the filesystem state data and mark any freed space as free.

The only downside I see of the BTRFS way of doing it is that you'll get more file fragmentation. But TBH, does fragmentation really make that much difference on most real systems these days ?




More information about the rsync mailing list