using virtual synchrony for CTDB

[tridge at samba.org]

Steven,

 > I'm not sure how it would be possible to lock a global resource without
 > communicating with all nodes that the resource should be locked.  One
 > way used in the current kernel DLM is to have "resource masters" where
 > control for a resource locking/unlocking is located on one particular
 > node that has an affinity for the resource.

yep, thats like the LMASTER, but CTDB allows it to float to other
nodes (the DMASTER) when the affinity changes.

 > This works well in the common case of no lock contention (because
 > there are no round trip replies) but does not work well in the case
 > of lock contention (because off-affinity nodes must request the
 > lock from the resource master).

and this is where CTDB really shines - I expect Samba on top of CTDB
will not actually need to ever have a lock held remotely. That is one
of the key ideas behind the conditional append call.

In fact, I expect some deployments will not use locking at all! If the
underlying OS and/or filesystem has a particularly bad fcntl() lock
implementation (eg. NFS) then we can disable all locking quite safely
as long as long as we don't use "LTDB bypass" (see the CTDB spec). In
that case we can use TDB_INTERNAL on the underlying tdb, and we become
completely indepedent of any OS constraints on locking.

 > It does not use epoll at the moment although that is something the
 > project has looked at.  Without any users of the totem library except
 > for openais, the need for enhancing the performance of poll is not
 > necessary for us since poll performs well for a small set of descriptors
 > which fits the openais model well.

epoll makes a big difference in Samba4. It is not uncommon for us to
have between 50 and 100 sockets active at one time in a process. In
that case epoll helps a lot!

Cheers, Tridge