[Samba] ctdb PARTIALLYONLINE

[steve]

On Sat, 2014-07-12 at 16:40 -0500, Jonn Taylor wrote:
> On 07/12/2014 10:55 AM, steve wrote:
> > On Sat, 2014-07-12 at 09:01 +0200, steve wrote:
> >> On Fri, 2014-07-11 at 23:50 +0200, steve wrote:
> >>> On Sat, 2014-07-12 at 06:58 +1100, me at electronico.nc wrote:
> >>>> Le 12/07/2014 03:40, steve a écrit :
> >>>>> drbd ctdb ocfs2
> >>>>> Hi
> >>>>> Everything seems OK apart from the IP takeover.
> >>>>>
> >>>>> public_addresses
> >>>>> 192.168.1.80/24 enp0s3
> >>>>> 192.168.1.81/24 enp0s3
> >>>>>
> >>>>> ctdb status
> >>>>> Number of nodes:2
> >>>>> pnn:0 192.168.1.10     PARTIALLYONLINE
> >>>>> pnn:1 192.168.1.11     PARTIALLYONLINE (THIS NODE)
> >>>>> Generation:2090727463
> >>>>> Size:2
> >>>>> hash:0 lmaster:0
> >>>>> hash:1 lmaster:1
> >>>>> Recovery mode:NORMAL (0)
> >>>>> Recovery master:1
> >>>>>
> >>>>> but we are getting:
> >>>>> 2014/07/11 18:17:37.313940 [recoverd:15925]: Failed to find node to
> >>>>> cover ip 192.168.1.81
> >>>>> 2014/07/11 18:17:37.314608 [recoverd:15925]: Failed to find node to
> >>>>> cover ip 192.168.1.80
> >>>>>
> >>>>> our nodes are on the other interface, enps08:
> >>>>> 192.168.1.10
> >>>>> 192.168.1.11
> >>>>>
> >>>>> At the moment, enp0s3 have fixed IPs:
> >>>>> 192.168.1.82
> >>>>> 192.168.1.83
> >>>>>
> >>>>> Question: how do we configure the interface for the public IPs?
> >>>>> Dow we need a third interface for the public ips?
> >>>>> Cheers,
> >>>>> Steve
> >>>>>
> >>>> Hi Steve,
> >>>> Do you have 2 NICs :
> >>>> enp0s3
> >>>> enps08
> >>>> ?
> >>>> Which one is LAN ?
> >>>> Is the other one only for HA ?
> >>>> Why have you used the same subnet on the 2 NICs ?
> >>>> Nicolas
> >>> Hi
> >>> enp0s3 LAN
> >>> enp0s8 ha
> >>>
> >>> We used the same subnet, otherwise, nothing showed up with ctdb ip.
> >>> (Initially we had the ha on 192.168.0.10 and 192.168.0.11)
> >>>
> >>> No idea how to configure the interface for the public IPs. We'd like the
> >>> lan IPs to be on the 192.168.1.x network if possible. We're joined to
> >>> the AD domain on that subnet.
> >>>
> >>> The HA nics are easy to configure with fixed IPs but how do we configure
> >>> the other card for the public IPs? ATM it's configured with fixed IP's.
> >>>
> >>> Any clues most gratefully received. I'm sure we've overlooked something
> >>> simple.
> >>> Cheers,
> >>> Steve
> >>>
> >>>
> >> OK. just one clue: would reading about 'bonding' be of any use?
> >>
> > Anyone?
> > Anyway, it looks like it is bonding so I created a bonding interface and
> > placed the public addresses there:
> > /etc/ctdb/public_addresses
> > 192.168.1.80/24 bond0
> > 192.168.1.81/24 bond0
> >
> > (we've also tried with enp0s3 in place of bond0)
> >
> > placed the nodes on their own subnet:
> >  /etc/ctdb/nodes
> > 192.168.0.10
> > 192.168.0.11
> >
> > and the enslaved the real interface with the bond. e.g. here is node 2:

> Look at the MAC addresses, bond0 and enp0s3 has the same MAC. Post the
> configs for your ethernet interfaces.

OK. Got it:
 # ctdb status
Number of nodes:2
pnn:0 192.168.0.10     OK (THIS NODE)
pnn:1 192.168.0.11     OK
Generation:1318343591
Size:2
hash:0 lmaster:0
hash:1 lmaster:1
Recovery mode:NORMAL (0)
Recovery master:0
 # ctdb ip
Public IPs on node 0
192.168.1.80 0
192.168.1.81 1

...and the failover is finally working:)
Thanks for the clues.

This is _so_ much better than DFS, but the documentation seems just as
misleading and inaccurate. Maybe I should put together a plain English
howto.

For the record and in true ha documentation style (with information
about the other nodes missing) here is node1:
/etc/sysconfig/network/ifcfg-bond0
BONDING_MASTER='yes'
BONDING_MODULE_OPTS='mode=balance-rr miimon=100'
BONDING_SLAVE0='enp0s3'
BOOTPROTO='static'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR='192.168.1.82/24'
MTU=''
NAME=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'

/etc/sysconfig/network/ifcfg-enp0s3
BOOTPROTO='none'
STARTMODE='hotplug'
NAME='82540EM Gigabit Ethernet Controller'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
USERCONTROL='no'

/etc/sysconfig/network/ifcfg-enp0s8
BOOTPROTO='static'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR='192.168.0.10/24'
MTU=''
NAME='82540EM Gigabit Ethernet Controller'
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'

Cheers,
Steve