[PATCHES] CTDB: improved IPv6 support
amitay at gmail.com
Fri Dec 5 06:39:31 MST 2014
On Thu, Dec 4, 2014 at 11:27 PM, Martin Schwenke <martin at meltin.net> wrote:
> On Thu, 4 Dec 2014 17:42:51 +1100, Martin Schwenke <martin at meltin.net>
> > The attached patch series fixes most problems with IPv6 in CTDB.
> > The most notable things are:
> > * For machine-readable output from the "ctdb" CLI tool, effectively
> > replace "ctdb -Y ..." with "ctdb -X ..." which uses '|' as the field
> > delimiter. For those that don't want '|', they can use
> > "ctdb -x<char> ...".
> > "ctdb -Y ..." is still supported but all documentation and scripts
> > now use "ctdb -X".
> > Mathieu, you're CC:ed because this includes a minor change to the
> > nagios script, which I think is correct but it isn't yet tested. :-)
> > * When the CTDB daemon loads the nodes file it stores a string
> > representation of each node's IP address. This representation is now
> > first converted to canonical form so that IPv6 addresses will
> > compare more reliably.
> > * Some functionality has moved from the daemon to the 10.interfaces
> > eventscript and it now works with IPv6 addresses.
> > For the past couple of years the release IP code in the daemon has
> > depended on being able to determine which interface an IP address is
> > on, rather than trusting vnn->interface. This was done to more
> > reliably be able to remove rogue IP addresses (present due to
> > either race conditions or addresses that have been moved by hand).
> > This is not as easy for IPv6 addresses since the SIOCGIFCONF ioctl
> > does not support IPv6.
> > Therefore, this code has been removed and vnn->interface is now
> > passed to the "releaseip" event, even when it is NULL (in which case
> > "__none__" is passed). In all cases the eventscript now uses some "ip
> > addr show to ..." magic to determine the actual interface and netmask
> > for an IP address being released. If either of these differ from
> > what was passed then a warning is logged, and the actual interface
> > and netmask are used to delete the IP address.
> > I don't think we lost any functionality. The main part of this is in
> > these patches:
> > 32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> > 837f36a ctdb-daemon: Trust vnn->interface for an IP when releasing it
> > * Gratuitous ARP equivalent for IPv6 has been changed to use neighbor
> > advertisements. This seems to work reliably.
> > * Due to duplicate address detection in IPv6, the IP address addition
> > code will try to wait until IPv6 addresses are no longer
> > "tentative". If this times out or "dadfail" occurs then the IP
> > address is removed and the operation fails. Note that this could
> > cause an outage (all nodes unhealthy, one at a time) if a rogue node
> > refuses to release IPs or if CTDB just dies. It isn't possible to
> > easily work around this, so it is a design limitation and may need to
> > be looked at again in the future.
> > This has been tested on a virtual cluster with IPv6 addresses only, on
> > both public and private networks.
> > Please review and push if OK.
> > These patches are in my ctdb-ipv6 branch:
> > Still to do:
> > * Update policy routing to work with IPv6. There's some code in there
> > that only works for IPv4.
> > * Other features like LVS.
> Darn. This looks like it was big enough to be moderated, so...
> These patches:
> b615d09f ctdb-tools: Produce machine readable output with new function
> fdf0461 ctdb-tools: Add -x option to specify delimiter for machine
> readable output
> 42498ee ctdb-tools: Add -X option for machine parsable output with
> separator '|'
> 8485235 ctdb-scripts: Update eventscripts to use ctdb -X instead of ctdb -Y
> 435c200 ctdb-tools: Update onnode and ctdb-diagnostics to use ctdb -X
> 8f1ebc5 ctdb-tests: Update integration tests to use ctdb -X
> 443aa32 ctdb-tool: Fix "ctdb -Y ifaces" output to have trailing delimiters
> c2e39e4 ctdb-doc: Update examples to use ctdb -X
> 1ea9d2e ctdb-utils: Update Nagios code to use ctdb -X
> c6a448c ctdb-scripts: Add IPv6 addresses support in ip_maskbits_iface()
> 9e6d323 ctdb-scripts: New functions ip6tables() and iptables_wrapper()
> 32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> 837f36a ctdb-daemon: Trust vnn->interface for an IP when releasing it
> b7debd7 ctdb-eventscripts: Specify broadcast optionally to ip addr add
> 38a5617 ctdb-scripts: Wait until IPv6 addresses are not "tentative"
> 7944a40 ctdb-daemon: Fix IP address comparisons for IPv6 addresses
> 51ec7d3 ctdb-tools: Bracket IP addresses in onnode (for IPv6)
> 9d4137d ctdb-tests: Extend regexp to match IPv6 addresses
> c9f3359 ctdb-tests: Try to handle IPv6 addresses for local daemons
In this patch, we cannot change the node IP addresses from 127.0.0.x to
127.0.234.x. Looks like socket wrapper does not like that, socket wrapper
expects addresses in 127.0.0.0/24 range. So here's a fixup to move node
IPs away from 127.0.0.1.
diff --git a/ctdb/tests/simple/scripts/local_daemons.bash
index 0131950..7d35a8f 100644
@@ -76,7 +76,8 @@ setup_ctdb ()
echo "fc00:10::1:$(($i + $TEST_LOCAL_DAEMONS))/64 lo"
- echo 127.0.234.$i >>"$CTDB_NODES"
+ j=$(( $i + 10))
+ echo 127.0.0.$j >>"$CTDB_NODES"
# 2 public addresses on most nodes, just to make things
if [ $(($i - 1)) -ne $no_public_ips ] ; then
echo "192.168.234.$i/24 lo" >>"$public_addresses_all"
> 875f60e ctdb-tests: Bracket IP addresses in NFS mounts and scp command
> (for IPv6)
> befdf53 ctdb-tests: Extend regexps to handle IPv6 address matching
> b2df1a3 ctdb-tests: Use ping_wrapper to do relevant ping or ping6
> 1cc75ea0 ctdb-tests: Match IPv6 connections in netstat output
> 928b88a ctdb-tests: Generalise the gratarp and tickle sniffing code for
> d546cba ctdb-tests: Use ip neigh command instead of arp
> 07b8aa1 ctdb-tests: Make tcpdump output more verbose
> 2ed1f6b ctdb-tests: More debug on SSH failure
> 76e83e5 ctdb-daemon: Gratuitous ARP equivalent for IPv6 is neighbor
> 242442a ctdb-tests: Need to drop public IPs in kill-failover tests
> from my ctdb-ipv6 branch, per above gitweb link.
> As an aside, does someone want to add me as a list moderator? ;-)
> peace & happiness,
I have pushed the extra patch and the rest of the patches with Reviewed-By
tag to ctdb-ipv6 branch in my tree.
More information about the samba-technical