[Samba] ctdb vacuum timeouts and record locks

[Martin Schwenke]

On Thu, 2 Nov 2017 12:17:56 -0700, Computerisms Corporation via samba
<samba at lists.samba.org> wrote:

> hm, I stand correct on the problem solved statement below.  Ip addresses 
> are simply not cooperating on the 2nd node.
> 
> root at vault1:~# ctdb ip
> Public IPs on node 0
> 192.168.120.90 0
> 192.168.120.91 0
> 192.168.120.92 0
> 192.168.120.93 0
> 
> root at vault2:/service/ctdb/log/main# ctdb ip
> Public IPs on node 1
> 192.168.120.90 0
> 192.168.120.91 0
> 192.168.120.92 0
> 192.168.120.93 0
> 
> root at vault2:/service/ctdb/log/main# ctdb moveip 192.168.120.90 1
> Control TAKEOVER_IP failed, ret=-1
> Failed to takeover IP on node 1
> 
> root at vault1:~# ctdb moveip 192.168.120.90 0
> Memory allocation error
> 
> root at vault2:/service/ctdb/log/main# ctdb ipinfo 192.168.120.90
> Public IP[192.168.120.90] info on node 1
> IP:192.168.120.90
> CurrentNode:0
> NumInterfaces:1
> Interface[1]: Name:eth0 Link:up References:0
> 
> Logs on vault2 (stays banned because it can't obtain IP):
> IP 192.168.120.90 still hosted during release IP callback, failing
> IP 192.168.120.92 still hosted during release IP callback, failing
> 
> root at vault1:~# ctdb delip 192.168.120.90
> root at vault1:~# ctdb delip 192.168.120.92
> root at vault2:/service/ctdb/log/main# ctdb addip 192.168.120.90/22 eth0
> Node already knows about IP 192.168.120.90
> root at vault2:/service/ctdb/log/main# ctdb ip
> Public IPs on node 1
> 192.168.120.90 -1
> 192.168.120.91 0
> 192.168.120.92 -1
> 192.168.120.93 0
> 
> 
> I am using the 10.external.  ip addr show shows the correct IP addresses 
> on eth0 in the lxc container.  rebooted the physical machine, this node 
> is buggered.  shut it down, used ip addr add to put the addresses on the 
> other node, used ctdb addip and the node took it and node1 is now 
> functioning with all 4 IPs just fine.  Or so it appears right now.
> 
> something is seriously schizophrenic here...

I'm wondering why you're using 10.external.  Although we have tested
it, we haven't actually seen it used in production before!  10.external
is a hack to allow use of CTDB's connection tracking while managing the
public IP addresses externally.  That is, you tell CTDB about the
public IPs, use "ctdb moveip" to inform CTDB about moved public IPs and
it sends grat ARPs and tickle ACKs on the takeover node.  It doesn't
actually assign the public IP addresses to nodes.

The documentation might not be clear on this but if you're using
10.external then you need to have the DisableIPFailover tunable set to
1 on all nodes so that CTDB doesn't try to move the IPs itself. 

Please let us know if the documentation could be improved... 

peace & happiness,
martin