CTDB IP takeover/failover tunables - do you use them?

[hvjunk]

> On 20 Apr 2017, at 07:16 , Martin Schwenke <martin at meltin.net> wrote:
> 
> On Wed, 19 Apr 2017 03:18:10 -0700 (PDT), Hvisage via samba-technical
> <samba-technical at lists.samba.org> wrote:
> 
>> Martin Schwenke wrote
>>> I'm currently hacking on CTDB's IP takeover/failover code.  For Samba
>>> 4.6, I would like to rationalise the IP takeover-related tunable
>>> parameters.
>>> 
>>> I would like to know if there are any users who set the values of these
>>> tunables to non-default values.  The tunables in question are:
>>> 
>>> ..snip..
>>> 
>>>   NoIPTakeover
>>>       Default: 0
>>> 
>>>       When set to 1, ctdb will not allow IP addresses to be failed over
>>> onto
>>>       this node. Any IP addresses that the node currently hosts will
>>> remain
>>>       on the node but no new IP addresses can be failed over to the node.
>>> 
>>> In particular, I would like to know if anyone has a use case where they
>>> set any of these variables to different values on different nodes.  This
>>> only really matters for the last 2 (NoIPHostOnAllDisabled,
>>> NoIPTakeover), since the value on the recovery master is just used for
>>> the other 2.  If you do this, can you please explain why?  :-)  
>> 
>> Sorry late reply, only saw this as I was implementing GlusterFS and first
>> had to battle SystemD..
>> 
>> My case:
>> glusterfs volume replica 3 arbiter1 NodeA NodeB NodeC
>> 
>> NodeA - volume VM on HyperVisorA in DC1 - prefer to have 10.0.1.1 on NodeA
>> NodeB - volume VM on HyperVisorB in DC2 - prefer to have 10.0.2.2 on NodeB
>> NodeC - Arbiter VM in the "cloud".
>> ClientA1, ClientA2 -VMs on HyperVisorA
>> ClientB1, ClientB2 - VMs on HypervisorB
>> 
>> I don't want NodeC to have the public IP, as it means a performance issue,
>> and when it's the only one available, ctdb would in any case be "down".
> 
> Right. So, in that case, you just set the public addresses list on
> NodeC to be empty.  You can even set
> CTDB_PUBLIC_ADDRESSES=/dev/null on NodeC - that's what our test suite
> does when it is running against "local daemons”.

Okay… the documentation I’ve read up till now, states something like:
 “the CTDB_PUBLIC_ADDRESSES file should be the exact same on all the nodes”,
 at least that’s how I interpreted it, so this just needs to be documented more explicitly ;)

> There are some corner cases where nodes with no public addresses will
> still get some unexpected messages sent to them and will log complaints
> at higher debug levels.  We will eventually get that sorted out.
> 
>>> There are also 2 tunables to choose the algorithm used to calculate the
>>> IP address layout:
>>> 
>>>   DeterministicIPs
>>>       Default: 0
>>> 
>>>       When set to 1, ctdb will try to keep public IP addresses locked to
>>>       specific nodes as far as possible. This makes it easier for
>>> debugging
>>>       since you can know that as long as all nodes are healthy public IP
>>> X
>>>       will always be hosted by node Y.  
>> 
>> Here my "use case" is that I want that to have ClientA1 & ClientA2 talk to
>> NodeA, and ClientB1 & ClientB2 talk to NodeB. The data is mostly reads, just
>> we need/want the uploads to be "stable", so I could point ClientB1 &
>> ClientB2 to 10.0.2.2 and ClientA1 & clientA2 to 10.0.1.1 and have the best
>> network performances.
>> 
>> I'm still looking/reading-docs as to how "deterministically" know that
>> 10.0.2.2 should be on NodeB & 10.0.1.1. should be on NodeA in the
>> normal/stable case?
> 
> You want to test deterministic IPs, which is now IPAllocAlgorithm=0 from
> Samba >= 4.6.0.
> 
> However, deterministics IPs do not (necessary) behave well when some
> nodes do not define public addresses.  The algorithm:
> 
> 1. Assigns addresses across all nodes in a modulo manner
> 
> 2. Drops addresses from unhealthy node
> 
> 3. Assigns unassigned addresses in a balanced manner
> 
> If you only have 2 public addresses and NodeA is node 0, NodeB is node
> 1 then it should work fine.  :-)

So, will the first public IP in the list be used, or the sorted list and that first IP to be deployed?
What will happen if the order of the public IPs differ on the nodes?

> 
> peace & happiness,
> martin