A Modest Proposal for Preventing the Event loops of Samba DCs From Being a Burthen to Their Implementers or Users, and for Making Them Beneficial to the Publick

[Simo]

On Thu, 2014-04-17 at 16:05 +1200, Andrew Bartlett wrote:
> I have a modest proposal that I think might just reduce the instances of
> DRS corruption and replication deadlocks we have been seeing recently,
> and move us one step closer the goal of not using nested event loops in
> Samba.
> 
> In debugging a recent case of a DRS deadlock, we found the DC running
> and looping while in a transaction.  Indeed, it isn't at all clear that
> the memory context it was using are valid, as it enters code that should
> be skipped due to having 'relax' set.
> 
> In any case, running other arbitrary things under a transaction can lead
> to quite strange errors, such as claiming the successful write of data
> that is then discarded when the transaction is cancelled.
> 
> (Transactions are rarely cancelled in Samba, which is why we got away
> with this for so long).
> 
> Anyway, what I propose is that when we we are in a transaction, that a
> new event context is created, and that this operates until the
> transaction is over.  That way, only internal ldb events should fire.
> (We never did define how non-nested transactions and async ldb would
> work in any case.) 
> 
> Other than LDB, the only other things using events at this point would
> be the irpc handler, which needs to be able to send messages to other
> parts of Samba.  Currently this uses it's own event context (an issue
> that will need to be fixed as we work to make that code common anyway),
> so this isn't affected/protected.
> 
> We could, and perhaps should, totally remove the use of an external
> event loop in Samba's invocation of LDB, but this step would make it
> harder to implement an async LDAP server is that was desired, but I see
> this as a useful first step.
> 
> It does make ldb less 'async' as far as the caller is concerned, but we
> simply don't use ldb in an async way in Samba.  (It is very unfortunate
> we carry the great complexity and risk of an async ldb without any
> significant use). 
> 
> I have this under a private autobuild, and I would appreciate your
> thoughts. 

I do not really see the point of a separate event context honestly.
All you need is clearly some locking, so that a new toplevel ldb
operation can only be started from within the transaction, while any
other is received but not scheduled until a previous transaction is
finished.
Blocking for long periods on heavy I/O in an async server will provide
terrible outcomes.

Simo.