[PATCHES] CTDB: improved IPv6 support
Amitay Isaacs
amitay at gmail.com
Fri Dec 5 07:16:21 MST 2014
On Sat, Dec 6, 2014 at 1:05 AM, Stefan (metze) Metzmacher <metze at samba.org>
wrote:
> Am 05.12.2014 um 15:02 schrieb Amitay Isaacs:
> > On Sat, Dec 6, 2014 at 12:42 AM, Stefan (metze) Metzmacher <
> metze at samba.org>
> > wrote:
> >
> >> Am 05.12.2014 um 14:39 schrieb Amitay Isaacs:
> >>> Hi Martin,
> >>>
> >>> On Thu, Dec 4, 2014 at 11:27 PM, Martin Schwenke <martin at meltin.net>
> >> wrote:
> >>>
> >>>> On Thu, 4 Dec 2014 17:42:51 +1100, Martin Schwenke <martin at meltin.net
> >
> >>>> wrote:
> >>>>
> >>>>> The attached patch series fixes most problems with IPv6 in CTDB.
> >>>>>
> >>>>> The most notable things are:
> >>>>>
> >>>>> * For machine-readable output from the "ctdb" CLI tool, effectively
> >>>>> replace "ctdb -Y ..." with "ctdb -X ..." which uses '|' as the
> field
> >>>>> delimiter. For those that don't want '|', they can use
> >>>>> "ctdb -x<char> ...".
> >>>>>
> >>>>> "ctdb -Y ..." is still supported but all documentation and scripts
> >>>>> now use "ctdb -X".
> >>>>>
> >>>>> Mathieu, you're CC:ed because this includes a minor change to the
> >>>>> nagios script, which I think is correct but it isn't yet tested.
> :-)
> >>>>>
> >>>>> * When the CTDB daemon loads the nodes file it stores a string
> >>>>> representation of each node's IP address. This representation is
> now
> >>>>> first converted to canonical form so that IPv6 addresses will
> >>>>> compare more reliably.
> >>>>>
> >>>>> * Some functionality has moved from the daemon to the 10.interfaces
> >>>>> eventscript and it now works with IPv6 addresses.
> >>>>>
> >>>>> For the past couple of years the release IP code in the daemon has
> >>>>> depended on being able to determine which interface an IP address
> is
> >>>>> on, rather than trusting vnn->interface. This was done to more
> >>>>> reliably be able to remove rogue IP addresses (present due to
> >>>>> either race conditions or addresses that have been moved by hand).
> >>>>> This is not as easy for IPv6 addresses since the SIOCGIFCONF ioctl
> >>>>> does not support IPv6.
> >>>>>
> >>>>> Therefore, this code has been removed and vnn->interface is now
> >>>>> passed to the "releaseip" event, even when it is NULL (in which
> case
> >>>>> "__none__" is passed). In all cases the eventscript now uses some
> >> "ip
> >>>>> addr show to ..." magic to determine the actual interface and
> netmask
> >>>>> for an IP address being released. If either of these differ from
> >>>>> what was passed then a warning is logged, and the actual interface
> >>>>> and netmask are used to delete the IP address.
> >>>>>
> >>>>> I don't think we lost any functionality. The main part of this is
> in
> >>>>> these patches:
> >>>>>
> >>>>> 32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> >>>>> 837f36a ctdb-daemon: Trust vnn->interface for an IP when
> releasing
> >> it
> >>>>>
> >>>>> * Gratuitous ARP equivalent for IPv6 has been changed to use neighbor
> >>>>> advertisements. This seems to work reliably.
> >>>>>
> >>>>> * Due to duplicate address detection in IPv6, the IP address addition
> >>>>> code will try to wait until IPv6 addresses are no longer
> >>>>> "tentative". If this times out or "dadfail" occurs then the IP
> >>>>> address is removed and the operation fails. Note that this could
> >>>>> cause an outage (all nodes unhealthy, one at a time) if a rogue
> node
> >>>>> refuses to release IPs or if CTDB just dies. It isn't possible to
> >>>>> easily work around this, so it is a design limitation and may need
> to
> >>>>> be looked at again in the future.
> >>>>>
> >>>>> This has been tested on a virtual cluster with IPv6 addresses only,
> on
> >>>>> both public and private networks.
> >>>>>
> >>>>> Please review and push if OK.
> >>>>>
> >>>>> These patches are in my ctdb-ipv6 branch:
> >>>>>
> >>>>>
> >>>>
> >>
> http://git.samba.org/?p=martins/samba.git;a=shortlog;h=refs/heads/ctdb-ipv6
> >>>>>
> >>>>> Still to do:
> >>>>>
> >>>>> * Update policy routing to work with IPv6. There's some code in
> there
> >>>>> that only works for IPv4.
> >>>>>
> >>>>> * Other features like LVS.
> >>>>
> >>>> Darn. This looks like it was big enough to be moderated, so...
> >>>>
> >>>> These patches:
> >>>>
> >>>> b615d09f ctdb-tools: Produce machine readable output with new function
> >>>> printm()
> >>>> fdf0461 ctdb-tools: Add -x option to specify delimiter for machine
> >>>> readable output
> >>>> 42498ee ctdb-tools: Add -X option for machine parsable output with
> >>>> separator '|'
> >>>> 8485235 ctdb-scripts: Update eventscripts to use ctdb -X instead of
> >> ctdb -Y
> >>>> 435c200 ctdb-tools: Update onnode and ctdb-diagnostics to use ctdb -X
> >>>> 8f1ebc5 ctdb-tests: Update integration tests to use ctdb -X
> >>>> 443aa32 ctdb-tool: Fix "ctdb -Y ifaces" output to have trailing
> >> delimiters
> >>>> c2e39e4 ctdb-doc: Update examples to use ctdb -X
> >>>> 1ea9d2e ctdb-utils: Update Nagios code to use ctdb -X
> >>>> c6a448c ctdb-scripts: Add IPv6 addresses support in
> ip_maskbits_iface()
> >>>> 9e6d323 ctdb-scripts: New functions ip6tables() and iptables_wrapper()
> >>>> 32c2eab ctdb-scripts: Make 10.interface IPv6-safe
> >>>> 837f36a ctdb-daemon: Trust vnn->interface for an IP when releasing it
> >>>> b7debd7 ctdb-eventscripts: Specify broadcast optionally to ip addr add
> >>>> 38a5617 ctdb-scripts: Wait until IPv6 addresses are not "tentative"
> >>>> 7944a40 ctdb-daemon: Fix IP address comparisons for IPv6 addresses
> >>>> 51ec7d3 ctdb-tools: Bracket IP addresses in onnode (for IPv6)
> >>>> 9d4137d ctdb-tests: Extend regexp to match IPv6 addresses
> >>>>
> >>>
> >>>
> >>>> c9f3359 ctdb-tests: Try to handle IPv6 addresses for local daemons
> >>>>
> >>>
> >>> In this patch, we cannot change the node IP addresses from 127.0.0.x to
> >>> 127.0.234.x. Looks like socket wrapper does not like that, socket
> >> wrapper
> >>> expects addresses in 127.0.0.0/24 range. So here's a fixup to move
> node
> >>> IPs away from 127.0.0.1.
> >>>
> >>> diff --git a/ctdb/tests/simple/scripts/local_daemons.bash
> >>> b/ctdb/tests/simple/scripts/local_daemons.bash
> >>> index 0131950..7d35a8f 100644
> >>> --- a/ctdb/tests/simple/scripts/local_daemons.bash
> >>> +++ b/ctdb/tests/simple/scripts/local_daemons.bash
> >>> @@ -76,7 +76,8 @@ setup_ctdb ()
> >>> echo "fc00:10::1:$(($i + $TEST_LOCAL_DAEMONS))/64 lo"
> >>>>> "$public_addresses_all"
> >>> fi
> >>> else
> >>> - echo 127.0.234.$i >>"$CTDB_NODES"
> >>> + j=$(( $i + 10))
> >>> + echo 127.0.0.$j >>"$CTDB_NODES"
> >>> # 2 public addresses on most nodes, just to make things
> >>> interesting.
> >>> if [ $(($i - 1)) -ne $no_public_ips ] ; then
> >>> echo "192.168.234.$i/24 lo" >>"$public_addresses_all"
> >>
> >> Maybe also the ipv6 socket_wrapper range:
> >>
> >> Samba uses this:
> >>
> >> $ctx->{ipv6} = sprintf("fd00:0000:0000:0000:0000:0000:5357:5f%02x",
> >> $swiface);
> >>
> >> metze
> >>
> >>
> > I don't see any hard-coding of specific IPv6 range in socket_wrapper
> code.
> > What am I missing?
>
> #ifdef HAVE_IPV6
> /*
> * FD00::5357:5FXX
> */
> static const struct in6_addr *swrap_ipv6(void)
> {
> static struct in6_addr v;
> static int initialized;
> int ret;
>
> if (initialized) {
> return &v;
> }
> initialized = 1;
>
> ret = inet_pton(AF_INET6, "FD00::5357:5F00", &v);
> if (ret <= 0) {
> abort();
> }
>
> return &v;
> }
>
Now I feel so stupid for searching fd00 in lower case! :-)
Another fixup patch pushed to ctdb-ipv6 branch in my tree.
Amitay.
More information about the samba-technical
mailing list