Latest TDB2 design and code...

[simo]

On Sun, 2010-09-12 at 21:24 +1000, tridge at samba.org wrote:
> Hi Ronnie,
> 
>  > Snapshots have many more nice properties than just ctdb and ctdb recoveries.
>  > 
>  > They would allow things like
>  > * rewind to content from previous snapshot
>  > * (if cheap) compute delta between snapshot x and snapshot y
>  > * compute delta between snapshot n and snapshot n-1 to allow backup or
>  > replication of deltas.
>  > * a series of deltas between n and n-1 allow for very compact
>  > representation of a series of point in time backups.
>  > * they provide an internally consistent point in time representations
>  > if the data, which could be used for backup
>  > or traversal purposes. traversing and/or backing the data up online
>  > without locking database.
> 
> yes, these are all basic properties of snapshots, but the question
> still stands - what will Samba use them for?
> 
> How will users benefit from us having snapshots?
> 
>  > Cheap snapshots have almost infinite number of use cases.
>  > I think snapshots are useful.
> 
> I love them in filesystems, but I also know just how much complexity
> they add and just how much they can affect performance and prevent
> optimisations.
> 
> I can see how they could be used in ctdb when you have wide area
> clusters. That is a pretty esoteric use case for something that will
> have a major impact on the code.
> 
> So what is another use case that would make it worthwhile to add this
> to tdb?

It may be interesting int he registry code if we add the twist that
taking a snapshot calls an fsync in a database otherwise marked as no
sync. This would allow us to basically "commit" a number of changes when
we have to (as required by MS docs), without suffering the SYNC penalty
all the time. On bad shutdown the recovery for this kind of DB could
automatically revert to the last good snapshot (at least if any error is
found). This would give us good performance (normally no-sync) and also
checkpoints at which we flush data (snapshots), without trying to
conflate this behavior in some sort of transaction that can't cope with
spanning multiple processes, and in general are not suited to this task
anyway.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Principal Software Engineer at Red Hat, Inc. <simo at redhat.com>