RAFT and CTDB

Michael Adam obnox at samba.org
Mon Dec 8 09:56:53 MST 2014


On 2014-12-08 at 16:26 +1100, Martin Schwenke wrote:
> On Sat, 6 Dec 2014 15:55:28 +1100, Martin Schwenke <martin at meltin.net>
> wrote:
> 
> > As various people have mentioned, the default filesystem requirement
> > is fcntl(2) locking support to support the CTDB recovery lock.
> > There's an assumption that if the ping_pong test succeeds then CTDB's
> > recovery lock will work.  If that's not true then we need to create a
> > new test.  However, I don't believe anyone has conclusively shown
> > ping_pong not to be a reliable test.
> 
> I rushed through some of this late on Saturday night and was actually
> fooled.  I thought the ping_pong test had passed and then CTDB failed
> in the same way that some people have described on this mailing list.
> 
> I have since updated the ping_pong wiki page at:
> 
>   https://wiki.samba.org/index.php/Ping_pong
> 
> by adding some bold and an extra section.
> 
> The summary is that you can't race through and simply confirm that the
> test prints the correct data_increment value when running with -rw.
> 
> For the recovery lock to work you need to run the non -rw version and
> actually confirm that *the locking rate drops dramatically*.  If it
> doesn't then it is *not* working!

This is not necessarily true!

For instance I remember that a few years ago, for GFS2 with the default
configuration, I observed a constant lock rate until I reached 5
nodes or so. This was due to the fact, that GFS' lock manager by
default restricted locks to 100/second. Only if you removed that
limit, you could see that dramatic drop.

Also the drop will not be as dramatic with every file system,
since file systems seem to have different levels of optimization
when only one node is involed.

I also remember (I think also with GFS), that initial lock rate
was pretty high for 1 node (with custom config), and dropped
drastically when I added a node. But when I removed the but-last
node, the rate did not raise as drastically as it initially
dropped, i.e. not to the orignal high lock rate.
The explanation was that the lock manager stayed in the special
mode for a single locking node only until a second locking node
was added, but it did not revert back to the special scheme
after the last had left (presumably based on a heuristic that
probably more lockers would come back later).

So I'd say that ping_pong without -rw is generally good for
seing possible lock rates, but if you want to verify real
behaviour, then you should test with -rw (of course only if
the file system implements coherence of data operations under
locks, which hopefully all file systems that we can seriously
take into account do...). :-)

Cheers - Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141208/89c73962/attachment.pgp>


More information about the samba-technical mailing list