using virtual synchrony for CTDB

[tridge at samba.org]

Tracy,

 > It is not clear that CTDB cares about message ordering at all (including 
 > the reqid seems sufficient)

right. Some of the consequences of that are interesting, but are well
within the requirements of the CIFS protocol.

For example, the lack of strict ordering in CTDB could mean that when
two CIFS clients connecting to the one Samba cluster node both open a
file, it is perfectly possible for the 2nd one trying to send the open
to get the file and the first one not (the first one receiving a
sharing violation due to the 2nd open).

The chances of this actually happening will be very small, but
non-zero. An easy way to see that this is OK is that it can in fact
happen now, even without a clustered server. It just requires that the
tcp socket from the first CIFS client be a bit slower in delivering
the data to smbd than the 2nd tcp socket (either due to network delays
or process scheduling).

Where things get more interesting is when both opens come from the
same CIFS client, and go to the same CTDB node. In that case the
ordering does matter, and the CTDB messaging client code will need to
provide a guaranteed ordering, but that can be done within that node,
with no need for the other CTDB nodes to be aware of that ordering
constraint.

It will be fun watching all this stuff with wireshark when we run our
CIFS test suite against the cluster :-)

Cheers, Tridge