Using TDB for glib/gobject applications

[simo]

On Wed, 2009-06-24 at 08:06 +1000, tridge at samba.org wrote:
> Hi Simo,
> 
>  > Concurrent transactions are basically impossible with TDB, how would you
>  > handle 2 separate transaction create an object with the same key ?
> 
> I wouldn't go as far as saying that they are impossible, but they are
> quite tricky. What you need is essentially what Ronnie and I added for
> the cluster-wide transactions in ctdb (see ctdb_replay_transaction()).

Sure, I meant impossible with the current interface/expectations.

> I've CCd Rusty as he is currently working on a possible new format and
> transaction system for tdb, so he may have some comments.
> 
> The way it works is this:
> 
>   - during a transaction, record in some data structure (eg. linked
>     list) all of the operations of the transaction. The key thing to
>     record is all the keys of the records that are read and the state
>     of the record. If the records had a sequence number that would be
>     ideal, otherwise a moderately strong hash of the record would do.
> 
>   - during the main transaction phase the locking would prevent
>     non-transaction writes from happening, but would not prevent other
>     transactions from starting.
> 
>   - during the commit phase an exclusive lock would be taken (so only
>     one commit happens at a time), and the commit logic would re-read
>     all the records that were read during the transaction. If any of
>     them have changed then we would have to fail the transaction. (for
>     ctdb we had to do something a bit more complex, where we had the
>     possibility of replaying the whole transaction. That was needed to
>     cope with network disconnects, which we don't have to worry about
>     for local tdb).
> 
> The big change would be that code that uses transaction would need to
> cope with the possibility of a transaction commit failing due to a
> conflict with another transaction. The usual way to cope with that is
> a retry loop. That was why I didn't put concurrent transactions in tdb
> - I wanted to keep the use of transactions simple from the programmers
> point of view.

Yes having to replay transactions is a big deal, esp. if you are taking
actions depending on the current status. It can be applied only in
situations where actions you take are easily reversible or do not cause
issues.

> One thing we could consider is having a tdb flag
> TDB_CONCURRENT_TRANSACTIONS to say that we are happy to cope with
> concurrent transactions. So existing code would get reliable
> transactions (reliable in the sense that you don't need to retry
> them), whereas places that need concurrent transactions could ask for
> them.

That would make a lot of sense.

> The main place I could see us using concurrent transactions in Samba
> would be ldb, where it might solve a problem we have at the moment,
> where the single process Samba4 process model can have multiple ldap
> 'transactions' at the ldb level mapped to a single tdb transaction. To
> solve I think that we'd also have to introduce the concept of a
> "transaction handle" to identify which transaction we're talking
> about, or find some way to have the same tdb open twice in the same
> process (perhaps similar to what Howard did for the split of tdb into
> per-thread and global parts of the tdb structure).

Not sure it would be a really good idea for ldb, serializing
transactions is probably ok, and will make a lot of uses much simpler.

Especially if you consider cases where we use an ldap backend.

There you cannot easily revert what you have done, so allowing a
transaction to fail for mere concurrency reasons is a recipe for
headaches, and possibly so difficult to solve to turn ldap backend into
being not usable in production.
I expect a much higher transaction failure rate in such a setup given an
ldap backend introduces also some latency, therefore expanding
considerably the window of opportunity for transaction clashes to
happen.

Another cause of trouble is the partitions module. What if half the tdbs
commit ok and then the last one fails to commit due to concurrency.
You would start to need a 2 phase transaction commit code ...

In LDB writes should be seldom enough that serializing transactions will
not cause any major problem, IMO.

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer <simo at samba.org>
Principal Software Engineer at Red Hat, Inc. <simo at redhat.com>