Parallel serving via NFS from CTDB considered harmful

[Jeff Layton]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've been playing with CTDB recently and ran across a couple of pages
that seem to indicate that serving a clustered filesystem from multiple
nodes using NFS is safe:

    http://ctdb.samba.org/nfs.html

...and...

    https://wiki.samba.org/index.php/CTDB_Setup#Setting_up_CTDB_for_clustered_NFS

I'm concerned that these pages are misleading since from what I can
tell, when a failover event occurs the lock recovery grace period is not
reinstated across the entire cluster.

The problematic scenario is something like this:

- - client1 mounts an exported filesystem from a public IP on serverA and
  acquires a lock on it

- - serverA crashes, its locks are released by DLM (or other clustered
lock manager)

- - client2 now races in and acquires a conflicting lock on serverB

- - the ip address now "floats" to serverC

- - client1 tries to reclaim his lock now, but can't

...even worse is a variant of the above case where client2 just briefly
gets the lock, modifies the data protected by it and then releases it.
client1 will think he's had exclusive access to the lock the whole
time. The problem in that case will manifest itself as silent data
corruption.

Getting clustered NFS locking right is really, really hard due to the
recovery semantics. I'd like to suggest that we either remove the pages
above, or at least add some warnings that locking may not be reliable
in such configurations.

Thoughts?
- -- 
Jeff Layton <jlayton at samba.org>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iQIcBAEBAgAGBQJQgo5wAAoJEAAOaEEZVoIVaBsP/1JdwzKqxUbm4zATkHdhfqbQ
MoZpdFQo6gxiWzJaQ1EkqPQoW20T9tOqtPWYZKiwe5OyX64Gvzpb7WSKleiiWFcE
TGTFEwJGzkplikwDt6FN3G4RuGm8zyPGKRbIKEEjGeaJ2768WEkOkmV3L1WMcCN+
xupCIFi4ukBQDfjrgNyrVYVAM1Qwe2SIMeN9YOQb+Ij0KgXTmXGoAMMsLe5QBv5I
q1bNOfqlMRQ/Hu/mOoxY5IG7q3hPZL7eV8bTmwx0osZ8iqBLz2JNAQSYTJvXX5Bu
m1MpgHH9rsEr1lsGnyLV4fMqo38laWbcilWR2YBY44wXzVMcaEJrbvG59KmqVT0h
W9IjhRUn5PJK9K7vLcQ9bWcHyGdFFnzMqzfDTp+KBBrpcv2lc/FSy3/zpVKGG/7X
7Wj6LsdFGdNbvDuRdqokqyz8wXfwUWxJY3RVmOhNe7K9sNdccXVsDNfFztVTchWW
wguZN8EfGV6mHtKbv4mCPNvyjTLHXL7wxCRHhMeTRK0E4aYMcePDiyooImmwMrDU
VclxRq23x1HYWHjT0Qv+BmNUBRA3BcwwrxyptouDUV3lypzYLtcpukSCKULBUqkt
tdy6q+5id+r4w/8A2NMda3v7aFDiVKzp0LB23gpEnu8bcZmoMGLIq+KBE5Yhiy20
7sVtUaj+dvPzaXRFz0xg
=cbZn
-----END PGP SIGNATURE-----