[clug] KVM Bridge networking - Update 2 - Now IPv6 stack never gets disabled, only enabled by systemd-networkd
George at Clug
Clug at goproject.info
Sat Apr 11 14:07:37 UTC 2020
On Saturday, 11-04-2020 at 19:54 Robert Edwards wrote:
> Hi George,
>
> I feel compelled to advise you to stop spamming the list with your
> investigations. You have made no attempt to explain what you are
> trying to do and so little possibility for anyone else to see how
> your investigations might apply to them.
I originally wrote "I wanted a KVM host server with an isolated
network bridge without IP
address configuration for both IPv4 and IPv6. I am interested if
anyone can explain how to achieve this.", which to me explains what I
am trying to do, and my question asking "how do I do this?"
Due to my error, my answer to Michael's question to me did not make
it to the CLUG list. (I hope the below more clearly explains my
original question?)
"I want "several isolated network bridges" on the KVM host, that are
isolated from both the host KVM server, and isolated from other the
bridges on the host KVM server.
The isolated network bridges are connected to NICs that are connected
to physical switches, and connecting to other physical and virtual
servers."
If the above does not clearly explain what I am asking, please ask me
to provide further detail about what it is you don't understand.
>
> If you have a specific question, please ask it.
I did ? why is it that you do not understand?
I originally wrote "I wanted a KVM host server with an isolated
network bridge without IP
address configuration for both IPv4 and IPv6. I am interested if
anyone can explain how to achieve this."
It is obvious to me, my question or request is "if you know how to do
the above, then please enlighten me".
I later asked "If you understand the below /etc/network/interfaces
lines, please let me know which lines you believe are correct, which
are incorrect and
why." It would be great to get an answer.
> If you want to share
> a heap of random research, please consider putting it up on a wiki
> or similar and send the list a short message with your URL and some
> explanation as to why it might be relevant to anyone else.
As to why I added the extra information,
1) it allows other to see my current train of thought,
2) what level of research I have attempted (previously people
complained about me asking questions without having done any research,
so hopefully this way they can see that I have indeed been
researching. 3) too often people have posted to CLUG but have not
given me enough information for me to understand what their issue is,
I prefer people to provide more detail, which I believe I did, so I am
confused you did not understand that I asked about I wanted a KVM host
server with an isolated network bridge without IP address
configuration for both IPv4 and IPv6."
3) if I provide links to sites that speak about the subject in
question, then someone might notice that I have missed other more
relevant information.
>
> Most people don't need their inboxes filled with pages of randomly
> assembled missives.
If my three emails (now four, with this email), can fill your inbox,
please delete these emails. Being text, they don't take up much space.
>
> Thanks, and good luck with your quest.
Thanks Bob, for your wishes, and unless someone answers my request,
then the best answer to date is that "I need to disable IPv6 for any
bridges on which I don't want an IPv6 addresses. Sadly there does not
seem to be a way to tell systemd.network not to assign an IPv6
address".
>
> cheers,
> Bob Edwards.
>
> On 11/4/20 4:21 pm, George at Clug via linux wrote:
> > Hi
> >
> > To anyone following this thread, please read on... (there is a lot
of links and reading so only "read on" is this subject is of keen
interest to you)
> >
> > It occurred to me if systemd.network was overriding
/etc/network/interfaces, and forcing IPv6 addressing onto the bridge
(e.g. br1), then maybe I could configure systemd.network not to do
this?
> >
> > Alas I was not successful, though I did find some interesting
reading material on systemd.network (systemd - I am amazed how one can
compound the simple)
> >
> > Most useful ideas
> >
https://manpages.debian.org/experimental/systemd/systemd.network.5.en.html
> > IPv6 will be automatically disabled for that interface by writing
"1" to /proc/sys/net/ipv6/conf/ifname/disable_ipv6.
> >
> > Things I tried to get systemd-networkd not to assign an IPv6
address to my bridge br1...
> >
> > Restart systemd-networkd service:
> > # systemctl daemon-reload
> > # systemctl restart systemd-networkd
> > # journalctl -b -u systemd-networkd
> >
> > # cat /etc/systemd/network/ens9.network
> > [Match]
> > Name = ens9
> >
> > [Network]
> > Description = For Bridge 1
> > DHCP = no
> > IPv6AcceptRA = no
> > # Enables link-local address autoconfiguration.
> > LinkLocalAddressing = no
> >
> > # find / -name *.network
> > /usr/lib/systemd/network/80-container-host0.network
> > /usr/lib/systemd/network/80-container-ve.network
> > /usr/lib/systemd/network/80-container-vz.network
> >
> > # ls -hal /lib/systemd/network
> > total 24K
> > drwxr-xr-x 2 root root 4.0K Apr 5 21:23 .
> > drwxr-xr-x 14 root root 4.0K Apr 5 21:23 ..
> > -rw-r--r-- 1 root root 645 Feb 14 2019
80-container-host0.network
> > -rw-r--r-- 1 root root 718 Feb 14 2019
80-container-ve.network
> > -rw-r--r-- 1 root root 704 Feb 14 2019
80-container-vz.network
> > -rw-r--r-- 1 root root 417 Feb 14 2019 99-default.link
> >
> > # grep -R "\[Network\]" /lib/systemd/network
> > /lib/systemd/network/80-container-host0.network:[Network]
> > /lib/systemd/network/80-container-ve.network:[Network]
> > /lib/systemd/network/80-container-vz.network:[Network]
> >
> > # ip a
> > 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen
1000
> > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > inet 127.0.0.1/8 scope host lo
> > valid_lft forever preferred_lft forever
> > inet6 ::1/128 scope host
> > valid_lft forever preferred_lft forever
> > 2: ens9:
mtu 1500 qdisc pfifo_fast master br1 state UP group default qlen 1000
> > link/ether 52:54:00:38:f8:68 brd ff:ff:ff:ff:ff:ff
> > 3: ens3:
mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
> > link/ether 52:54:00:d9:60:82 brd ff:ff:ff:ff:ff:ff
> > inet 10.0.0.122/24 brd 10.0.0.255 scope global ens3
> > valid_lft forever preferred_lft forever
> > inet6 fe80::5054:ff:fed9:6082/64 scope link
> > valid_lft forever preferred_lft forever
> > 4: br1:
mtu 1500 qdisc noqueue state UP group default qlen 1000
> > link/ether 52:54:00:38:f8:68 brd ff:ff:ff:ff:ff:ff
> > inet6 fe80::5054:ff:fe38:f868/64 scope link
> > valid_lft forever preferred_lft forever
> >
> >
> >
> >
> > The below link indicated to me that I was embarking on a futile
journey when it comes to getting systemd.network to not enforce a IPv6
address on my bridge.
> >
https://unix.stackexchange.com/questions/544749/how-to-fully-disable-ipv6-in-lxd-containers-with-systemd-networkd
> >
> > I am aware of sysctl settings to disable IPv6, which will work,
but I would prefer to not use these as I don't want to disable IPv6
globally, and the virtual adaptors may be re-configured to use IPv6 or
not dynamically.
> >
> > That behavior changed in 482efedc08 for no apparent reason, docs
weren't updated. Now IPv6 stack never gets disabled, only enabled by
systemd-networkd.
> >
> >
https://github.com/systemd/systemd/commit/482efedc081b0c4bf2e77a3dee6b979d9c9a5765#diff-12a3ecdf5af7aa9f8583c27ac442ca43
> >
> >
> >
> > The links below seemed to be useful in understanding the futility
of my efforts.
> >
> >
https://coreos.com/os/docs/latest/network-config-with-networkd.html
> > Turn off IPv6 on specific interfaces
> >
> > While IPv6 can be disabled globally at boot by appending
ipv6.disable=1 to the kernel command line, networkd supports disabling
IPv6 on a per-interface basis. When a network unit's [Network] section
has either LinkLocalAddressing=ipv4 or LinkLocalAddressing=no,
networkd will not try to configure IPv6 on the matching interfaces.
> >
> > Note however that even when using the above option, networkd will
still be expecting to receive router advertisements if IPv6 is not
disabled globally. If IPv6 traffic is not being received by the
interface (e.g. due to sysctl or ip6tables settings), it will remain
in the configuring state and potentially cause timeouts for services
waiting for the network to be fully configured. To avoid this, the
IPv6AcceptRA=no option should also be set in the [Network] section.
> >
> > A network unit file's [Network] section should therefore contain
the following to disable IPv6 on its matching interfaces.
> >
> > [Network]
> > LinkLocalAddressing=no
> > IPv6AcceptRA=no
> >
> > And restart systemd-networkd service:
> >
> > systemctl daemon-reload
> > systemctl restart systemd-networkd
> > journalctl -b -u systemd-networkd
> >
> >
> >
https://manpages.debian.org/experimental/systemd/systemd.network.5.en.html
> > IPv6 will be automatically disabled for that interface by writing
"1" to /proc/sys/net/ipv6/conf/ifname/disable_ipv6.
> >
> > The .network files are read from the files located in the system
network directories /lib/systemd/network and
/usr/local/lib/systemd/network, the volatile runtime network directory
/run/systemd/network and the local administration network directory
/etc/systemd/network.
> >
> > NETWORK] SECTION OPTIONS
> > The "[Network]" section accepts the following keys:
> >
> > LinkLocalAddressing=
> > Enables link-local address autoconfiguration. Accepts "yes", "no",
"ipv4", "ipv6", "fallback", or "ipv4-fallback". If "fallback" or
"ipv4-fallback" is specified, then an IPv4 link-local address is
configured only when DHCPv4 fails. If "fallback", an IPv6 link-local
address is always configured, and if "ipv4-fallback", the address is
not configured. Note that, the fallback mechanism works only when
DHCPv4 client is enabled, that is, it requires "DHCP=yes" or
"DHCP=ipv4". If Bridge= is set, defaults to "no", and if not, defaults
to "ipv6".
> >
> > IPv6AcceptRA=
> > Takes a boolean. Controls IPv6 Router Advertisement (RA) reception
support for the interface. If true, RAs are accepted; if false, RAs
are ignored, independently of the local forwarding state.
> >
> > [IPV6PREFIX] SECTION OPTIONS¶
> > One or more "[IPv6Prefix]" sections contain the IPv6 prefixes that
are announced via Router Advertisements. See RFC 4861[18] for further
details.
> >
> > AddressAutoconfiguration=, OnLink=
> > Takes a boolean to specify whether IPv6 addresses can be
autoconfigured with this prefix and whether the prefix can be used for
onlink determination. Both settings default to "true" in order to ease
configuration.
> >
> >
https://www.sciencedirect.com/topics/computer-science/autoconfiguration
> > http://6lab.cz/ipv6-autoconfiguration/
> >
https://www.freedesktop.org/software/systemd/man/systemd.network.html
> >
> > https://github.com/systemd/systemd/issues/12917
> >
> >
https://manpages.debian.org/stretch/systemd/systemd.network.5.en.html
> > NAME
> > systemd.network - Network configuration
> > SYNOPSIS
> > network.network
> >
> > LinkLocalAddressing=
> > Enables link-local address autoconfiguration. Accepts "yes", "no",
"ipv4", "ipv6", "fallback", or "ipv4-fallback". If "fallback" or
"ipv4-fallback" is specified, then an IPv4 link-local address is
configured only when DHCPv4 fails. If "fallback", an IPv6 link-local
address is always configured, and if "ipv4-fallback", the address is
not configured. Note that, the fallback mechanism works only when
DHCPv4 client is enabled, that is, it requires "DHCP=yes" or
"DHCP=ipv4". If Bridge= is set, defaults to "no", and if not, defaults
to "ipv6".
> >
> > IPv6AcceptRA=
> > Takes a boolean. Controls IPv6 Router Advertisement (RA) reception
support for the interface. If true, RAs are accepted; if false, RAs
are ignored, independently of the local forwarding state. When RAs are
accepted, they may trigger the start of the DHCPv6 client if the
relevant flags are set in the RA data, or if no routers are found on
the link.
> >
> > Further settings for the IPv6 RA support may be configured in the
"[IPv6AcceptRA]" section, see below.
> >
> > Also see ip-sysctl.txt[6] in the kernel documentation regarding
"accept_ra", but note that systemd's setting of 1 (i.e. true)
corresponds to kernel's setting of 2.
> >
> > Note that kernel's implementation of the IPv6 RA protocol is
always disabled, regardless of this setting. If this option is
enabled, a userspace implementation of the IPv6 RA protocol is used,
and the kernel's own implementation remains disabled, since
systemd-networkd needs to know all details supplied in the
advertisements, and these are not available from the kernel if the
kernel's own implementation is used.
> >
> >
> >
> > # cat /etc/network/interfaces
> > # This file describes the network interfaces available on your
system
> > # and how to activate them. For more information, see
interfaces(5).
> >
> > source /etc/network/interfaces.d/*
> >
> > # The loopback network interface
> > auto lo
> > iface lo inet loopback
> >
> > #VERBOSE="yes"
> >
> > #DEBUG="yes"
> >
> > # SYSLOG="yes"
> >
> > # The primary network interface
> > auto ens3
> > #allow-hotplug ens3
> > iface ens3 inet static
> > address 10.0.0.122/24
> > gateway 10.0.0.254
> > # dns-* options are implemented by the resolvconf package, if
installed
> > dns-nameservers 10.0.0.254
> > dns-search goproject.info
> >
> > iface ens9 inet manual
> >
> > #auto br1
> > #iface br1 inet manual
> > # bridge_ports ens9
> > # up /usr/sbin/brctl setageing br1 0
> > # up /usr/sbin/brctl stp br1 off
> > #
> > #iface ens9 inet6 manual
> > # bridge_ports ens9
> > # up /usr/sbin/brctl setageing br1 0
> > # up /usr/sbin/brctl stp br1 off
> >
> > auto br1
> > iface br1 inet manual
> > bridge_ports ens9
> > autoconf 0
> > accept_ra 0
> > bridge_stp off
> > bridge_fd 0
> > bridge_waitport 0
> >
> > iface br1 inet6 manual
> > bridge_ports ens9
> > autoconf 0
> > accept_ra 0
> > bridge_stp off
> > bridge_waitport 0
> > bridge_fd 0
> >
> >
> >
> >
> > On Saturday, 11-04-2020 at 12:07 George at Clug via linux wrote:
> >> Hi,
> >>
> >> In my quest for isolated bridges for a KVM host, I have had some
success, but only by disabling IPv6 on the bridge itself (thanks Bob
for the suggestion of disabling IPv6).
> >>
> >> If you understand this stuff better than myself, please point out
any stupid errors may have made. (this has been a four day
intensive research journey for myself).
> >>
> >> I would prefer not to have had to disable IPv6 on the bridge, but
until I learn how to stop "systemd-networkd" from re-enabling my
efforts to disable autoconfiguration on the bridge, it will have to do
as a solution.
> >>
> >> First I found this statement, which if true explains why I was
not able to disable IPv6 addresses, even after setting IPv6
autoconfiguration as disabled.
> >>
> >> "The problem with Ubuntu 18 and ipv6 is that systemd-networkd
controls kernel parameters, so though one might disable ipv6 with
sysctl, networkd will be more than happy switching them on for you, if
the configuration does not state otherwise."
> >>
> >> The below web page provided some help ful suggestions
> >> https://hsmr.cc/Freifunk/GatewayKonfiguration
> >>
> >> /etc/sysctl.conf
> >> Uncomment
> >> net.ipv4.ip_forward=1
> >> net.ipv6.conf.all.forwarding=1
> >> net.ipv4.conf.default.rp_filter=1
> >> net.ipv4.conf.all.rp_filter=1
> >> Append
> >> # Disable IPv6 autoconf
> >> net.ipv6.conf.all.autoconf = 0
> >> net.ipv6.conf.default.autoconf = 0
> >> net.ipv6.conf.eth0.autoconf = 0
> >>
> >> net.ipv6.conf.all.accept_ra = 0
> >> net.ipv6.conf.default.accept_ra = 0
> >> net.ipv6.conf.eth0.accept_ra = 0
> >>
> >> # Do not process traffic on bridges with
iptables
> >> net.bridge.bridge-nf-call-arptables = 0
> >> net.bridge.bridge-nf-call-ip6tables = 0
> >> net.bridge.bridge-nf-call-iptables = 0
> >>
> >>
> >> I tried this approach but it did not work for me, I guess because
of the "systemd-networkd controls kernel parameters" issue.
> >>
> >> Not knowing how to stop systemd-networkd from overriding settings
or how to get systemd-networkd to do what I wanted to achieve, I keep
researching, and while it is going further than I wanted to do,
disabling IPv6 on the bridge does appear to acheive what I wanted to
do.
> >>
> >>
https://superuser.com/questions/575684/how-to-disable-ipv6-on-a-specific-interface-in-linux/575701
> >> You can disable it from /etc/sysctl.conf with this line:
> >> net.ipv6.conf.eth0.disable_ipv6 = 1
> >> Take a look at /proc/sys/net/ipv6/conf/eth0. There are many
options you can set in that directory, like leaving IPv6 enabled but
disabling autoconf etc.
> >>
> >> Disabling IPv6 for the specific bridges that I do not want IPv6
addresses, seems to have worked, and IPv6 traffic is forwarded from
the physical network via the NIC to the VMs connected to the bridge
(tested using ping6), while I cannot ping between bridges (which is
what I do want).
> >>
> >>
> >> Below are some other links I reviewed.
> >> ==========================================
> >>
> >> bridge: filtering via arp/ip/ip6tables is no longer available by
default. Update your scripts to load br_netfilter if you need this.
> >>
> >> https://ycnrg.org/vga-passthrough-with-ovmf-vfio/
> >>
> >> https://hsmr.cc/Freifunk/GatewayKonfiguration
> >>
> >>
https://freifunk-herford.github.io/ffhf-salt/file_roots/gateway/network/index.html
> >>
> >> =================================================================
> >>
https://superuser.com/questions/33196/how-to-disable-autoconfiguration-on-ipv6-in-linux
> >>
============================================================================
> >> # apt install dnsutils net-tools bridge-utils -y
> >>
> >> # sysctl -w net.ipv6.conf.br1.disable_ipv6=1
> >>
> >> # nano /etc/sysctl.conf
> >> Uncomment
> >> net.ipv4.ip_forward=1
> >> net.ipv6.conf.all.forwarding=1
> >> net.ipv4.conf.default.rp_filter=1
> >> net.ipv4.conf.all.rp_filter=1
> >> Append
> >> # Disable IPv6 autoconf
> >> net.ipv6.conf.all.autoconf = 0
> >> net.ipv6.conf.default.autoconf = 0
> >> net.ipv6.conf.eth0.autoconf = 0
> >>
> >> net.ipv6.conf.all.accept_ra = 0
> >> net.ipv6.conf.default.accept_ra = 0
> >> net.ipv6.conf.eth0.accept_ra = 0
> >>
> >> # Do not process traffic on bridges with
iptables
> >> net.bridge.bridge-nf-call-arptables = 0
> >> net.bridge.bridge-nf-call-ip6tables = 0
> >> net.bridge.bridge-nf-call-iptables = 0
> >>
> >> #Disable IPv6 for bridge
> >> net.ipv6.conf.br1.disable_ipv6=1
> >>
> >> # ls /proc/sys/net/ipv6/conf/
> >> # ls /proc/sys/net/ipv6/conf/br1/
> >> # ls /proc/sys/net/ipv6/conf/br1/autoconf
> >> # cat /proc/sys/net/ipv6/conf/br1/autoconf
> >> # cat /proc/sys/net/ipv6/conf/br1/accept_ra
> >> # cat /proc/sys/net/ipv6/conf/br1/disable_ipv6
> >> # cat /proc/sys/net/ipv6/conf/br1/use_tempaddr
> >> # cat /proc/sys/net/ipv6/conf/br1/use_oif_addrs_only
> >> # cat /proc/sys/net/ipv6/conf/br1/forwarding
> >>
> >>
> >>
> >> ================================================================
> >> # This file describes the network interfaces available on your
system
> >> # and how to activate them. For more information, see
interfaces(5).
> >>
> >> source /etc/network/interfaces.d/*
> >>
> >> # The loopback network interface
> >> auto lo
> >> iface lo inet loopback
> >>
> >> #VERBOSE="yes"
> >>
> >> #DEBUG="yes"
> >>
> >> # SYSLOG="yes"
> >>
> >> # The primary network interface
> >> auto ens3
> >> #allow-hotplug ens3
> >> iface ens3 inet static
> >> address 10.0.0.122/24
> >> gateway 10.0.0.254
> >> # dns-* options are implemented by the resolvconf package, if
installed
> >> dns-nameservers 10.0.0.254
> >> dns-search goproject.info
> >>
> >> iface ens9 inet manual
> >>
> >> #auto br1
> >> #iface br1 inet manual
> >> # bridge_ports ens9
> >> # up /usr/sbin/brctl setageing br1 0
> >> # up /usr/sbin/brctl stp br1 off
> >> #
> >> #iface ens9 inet6 manual
> >> # bridge_ports ens9
> >> # up /usr/sbin/brctl setageing br1 0
> >> # up /usr/sbin/brctl stp br1 off
> >>
> >> auto br1
> >> iface br1 inet manual
> >> bridge_ports ens9
> >> autoconf 0
> >> accept_ra 0
> >> bridge_stp off
> >> bridge_fd 0
> >> bridge_waitport 0
> >>
> >> iface br1 inet6 manual
> >> bridge_ports ens9
> >> autoconf 0
> >> accept_ra 0
> >> bridge_stp off
> >> bridge_waitport 0
> >> bridge_fd 0
> >>
> >>
> >> --
> >> linux mailing list
> >> linux at lists.samba.org
> >> https://lists.samba.org/mailman/listinfo/linux
> >>
> >
>
>
More information about the linux
mailing list