[PATCHES] CTDB: improved IPv6 support

[Amitay Isaacs]

On Sat, Dec 6, 2014 at 12:42 AM, Stefan (metze) Metzmacher <metze at samba.org>
wrote:

> Am 05.12.2014 um 14:39 schrieb Amitay Isaacs:
> > Hi Martin,
> >
> > On Thu, Dec 4, 2014 at 11:27 PM, Martin Schwenke <martin at meltin.net>
> wrote:
> >
> >> On Thu, 4 Dec 2014 17:42:51 +1100, Martin Schwenke <martin at meltin.net>
> >> wrote:
> >>
> >>> The attached patch series fixes most problems with IPv6 in CTDB.
> >>>
> >>> The most notable things are:
> >>>
> >>> * For machine-readable output from the "ctdb" CLI tool, effectively
> >>>   replace "ctdb -Y ..." with "ctdb -X ..." which uses '|' as the field
> >>>   delimiter.  For those that don't want '|', they can use
> >>>   "ctdb -x<char> ...".
> >>>
> >>>   "ctdb -Y ..." is still supported but all documentation and scripts
> >>>   now use "ctdb -X".
> >>>
> >>>   Mathieu, you're CC:ed because this includes a minor change to the
> >>>   nagios script, which I think is correct but it isn't yet tested. :-)
> >>>
> >>> * When the CTDB daemon loads the nodes file it stores a string
> >>>   representation of each node's IP address.  This representation is now
> >>>   first converted to canonical form so that IPv6 addresses will
> >>>   compare more reliably.
> >>>
> >>> * Some functionality has moved from the daemon to the 10.interfaces
> >>>   eventscript and it now works with IPv6 addresses.
> >>>
> >>>   For the past couple of years the release IP code in the daemon has
> >>>   depended on being able to determine which interface an IP address is
> >>>   on, rather than trusting vnn->interface.  This was done to more
> >>>   reliably be able to remove rogue IP addresses (present due to
> >>>   either race conditions or addresses that have been moved by hand).
> >>>   This is not as easy for IPv6 addresses since the SIOCGIFCONF ioctl
> >>>   does not support IPv6.
> >>>
> >>>   Therefore, this code has been removed and vnn->interface is now
> >>>   passed to the "releaseip" event, even when it is NULL (in which case
> >>>   "__none__" is passed).  In all cases the eventscript now uses some
> "ip
> >>>   addr show to ..." magic to determine the actual interface and netmask
> >>>   for an IP address being released.  If either of these differ from
> >>>   what was passed then a warning is logged, and the actual interface
> >>>   and netmask are used to delete the IP address.
> >>>
> >>>   I don't think we lost any functionality.  The main part of this is in
> >>>   these patches:
> >>>
> >>>     32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> >>>     837f36a ctdb-daemon: Trust vnn->interface for an IP when releasing
> it
> >>>
> >>> * Gratuitous ARP equivalent for IPv6 has been changed to use neighbor
> >>>   advertisements.  This seems to work reliably.
> >>>
> >>> * Due to duplicate address detection in IPv6, the IP address addition
> >>>   code will try to wait until IPv6 addresses are no longer
> >>>   "tentative".  If this times out or "dadfail" occurs then the IP
> >>>   address is removed and the operation fails.  Note that this could
> >>>   cause an outage (all nodes unhealthy, one at a time) if a rogue node
> >>>   refuses to release IPs or if CTDB just dies.  It isn't possible to
> >>>   easily work around this, so it is a design limitation and may need to
> >>>   be looked at again in the future.
> >>>
> >>> This has been tested on a virtual cluster with IPv6 addresses only, on
> >>> both public and private networks.
> >>>
> >>> Please review and push if OK.
> >>>
> >>> These patches are in my ctdb-ipv6 branch:
> >>>
> >>>
> >>
> http://git.samba.org/?p=martins/samba.git;a=shortlog;h=refs/heads/ctdb-ipv6
> >>>
> >>> Still to do:
> >>>
> >>> * Update policy routing to work with IPv6.  There's some code in there
> >>>   that only works for IPv4.
> >>>
> >>> * Other features like LVS.
> >>
> >> Darn.  This looks like it was big enough to be moderated, so...
> >>
> >> These patches:
> >>
> >> b615d09f ctdb-tools: Produce machine readable output with new function
> >> printm()
> >> fdf0461 ctdb-tools: Add -x option to specify delimiter for machine
> >> readable output
> >> 42498ee ctdb-tools: Add -X option for machine parsable output with
> >> separator '|'
> >> 8485235 ctdb-scripts: Update eventscripts to use ctdb -X instead of
> ctdb -Y
> >> 435c200 ctdb-tools: Update onnode and ctdb-diagnostics to use ctdb -X
> >> 8f1ebc5 ctdb-tests: Update integration tests to use ctdb -X
> >> 443aa32 ctdb-tool: Fix "ctdb -Y ifaces" output to have trailing
> delimiters
> >> c2e39e4 ctdb-doc: Update examples to use ctdb -X
> >> 1ea9d2e ctdb-utils: Update Nagios code to use ctdb -X
> >> c6a448c ctdb-scripts: Add IPv6 addresses support in ip_maskbits_iface()
> >> 9e6d323 ctdb-scripts: New functions ip6tables() and iptables_wrapper()
> >> 32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> >> 837f36a ctdb-daemon: Trust vnn->interface for an IP when releasing it
> >> b7debd7 ctdb-eventscripts: Specify broadcast optionally to ip addr add
> >> 38a5617 ctdb-scripts: Wait until IPv6 addresses are not "tentative"
> >> 7944a40 ctdb-daemon: Fix IP address comparisons for IPv6 addresses
> >> 51ec7d3 ctdb-tools: Bracket IP addresses in onnode (for IPv6)
> >> 9d4137d ctdb-tests: Extend regexp to match IPv6 addresses
> >>
> >
> >
> >> c9f3359 ctdb-tests: Try to handle IPv6 addresses for local daemons
> >>
> >
> > In this patch, we cannot change the node IP addresses from 127.0.0.x to
> > 127.0.234.x.  Looks like socket wrapper does not like that, socket
> wrapper
> > expects addresses in 127.0.0.0/24 range.  So here's a fixup to move node
> > IPs away from 127.0.0.1.
> >
> > diff --git a/ctdb/tests/simple/scripts/local_daemons.bash
> > b/ctdb/tests/simple/scripts/local_daemons.bash
> > index 0131950..7d35a8f 100644
> > --- a/ctdb/tests/simple/scripts/local_daemons.bash
> > +++ b/ctdb/tests/simple/scripts/local_daemons.bash
> > @@ -76,7 +76,8 @@ setup_ctdb ()
> >                 echo "fc00:10::1:$(($i + $TEST_LOCAL_DAEMONS))/64 lo"
> >>> "$public_addresses_all"
> >             fi
> >         else
> > -           echo 127.0.234.$i >>"$CTDB_NODES"
> > +           j=$(( $i + 10))
> > +           echo 127.0.0.$j >>"$CTDB_NODES"
> >             # 2 public addresses on most nodes, just to make things
> > interesting.
> >             if [ $(($i - 1)) -ne $no_public_ips ] ; then
> >                 echo "192.168.234.$i/24 lo" >>"$public_addresses_all"
>
> Maybe also the ipv6 socket_wrapper range:
>
> Samba uses this:
>
> $ctx->{ipv6} = sprintf("fd00:0000:0000:0000:0000:0000:5357:5f%02x",
> $swiface);
>
> metze
>
>
I don't see any hard-coding of specific IPv6 range in socket_wrapper code.
What am I missing?

Amitay.