CTDB IP takeover/failover tunables - do you use them?

Martin Schwenke martin at meltin.net
Thu Apr 20 05:16:14 UTC 2017

On Wed, 19 Apr 2017 03:18:10 -0700 (PDT), Hvisage via samba-technical
<samba-technical at lists.samba.org> wrote:

> Martin Schwenke wrote
> > I'm currently hacking on CTDB's IP takeover/failover code.  For Samba
> > 4.6, I would like to rationalise the IP takeover-related tunable
> > parameters.
> > 
> > I would like to know if there are any users who set the values of these
> > tunables to non-default values.  The tunables in question are:
> > 
> > ..snip..
> > 
> >    NoIPTakeover
> >        Default: 0
> > 
> >        When set to 1, ctdb will not allow IP addresses to be failed over
> > onto
> >        this node. Any IP addresses that the node currently hosts will
> > remain
> >        on the node but no new IP addresses can be failed over to the node.
> > 
> > In particular, I would like to know if anyone has a use case where they
> > set any of these variables to different values on different nodes.  This
> > only really matters for the last 2 (NoIPHostOnAllDisabled,
> > NoIPTakeover), since the value on the recovery master is just used for
> > the other 2.  If you do this, can you please explain why?  :-)  
> Sorry late reply, only saw this as I was implementing GlusterFS and first
> had to battle SystemD..
> My case:
> glusterfs volume replica 3 arbiter1 NodeA NodeB NodeC
> NodeA - volume VM on HyperVisorA in DC1 - prefer to have on NodeA
> NodeB - volume VM on HyperVisorB in DC2 - prefer to have on NodeB
> NodeC - Arbiter VM in the "cloud".
> ClientA1, ClientA2 -VMs on HyperVisorA
> ClientB1, ClientB2 - VMs on HypervisorB
> I don't want NodeC to have the public IP, as it means a performance issue,
> and when it's the only one available, ctdb would in any case be "down".

Right. So, in that case, you just set the public addresses list on
NodeC to be empty.  You can even set
CTDB_PUBLIC_ADDRESSES=/dev/null on NodeC - that's what our test suite
does when it is running against "local daemons".

There are some corner cases where nodes with no public addresses will
still get some unexpected messages sent to them and will log complaints
at higher debug levels.  We will eventually get that sorted out.

> > There are also 2 tunables to choose the algorithm used to calculate the
> > IP address layout:
> > 
> >    DeterministicIPs
> >        Default: 0
> > 
> >        When set to 1, ctdb will try to keep public IP addresses locked to
> >        specific nodes as far as possible. This makes it easier for
> > debugging
> >        since you can know that as long as all nodes are healthy public IP
> > X
> >        will always be hosted by node Y.  
> Here my "use case" is that I want that to have ClientA1 & ClientA2 talk to
> NodeA, and ClientB1 & ClientB2 talk to NodeB. The data is mostly reads, just
> we need/want the uploads to be "stable", so I could point ClientB1 &
> ClientB2 to and ClientA1 & clientA2 to and have the best
> network performances.
> I'm still looking/reading-docs as to how "deterministically" know that
> should be on NodeB & should be on NodeA in the
> normal/stable case?

You want to test deterministic IPs, which is now IPAllocAlgorithm=0 from
Samba >= 4.6.0.

However, deterministics IPs do not (necessary) behave well when some
nodes do not define public addresses.  The algorithm:

1. Assigns addresses across all nodes in a modulo manner

2. Drops addresses from unhealthy node

3. Assigns unassigned addresses in a balanced manner

If you only have 2 public addresses and NodeA is node 0, NodeB is node
1 then it should work fine.  :-)

peace & happiness,

More information about the samba-technical mailing list