[PATCH] selftest: Add support for running tests using linux namespaces
Tim Beale
timbeale at catalyst.net.nz
Fri May 31 02:27:50 UTC 2019
This patch-set adds optional/experimental support for using linux
namespaces instead of socket-wrapper.
The namespaces functionality is fairly self-contained in a few new
scripts, and we only need to hook into a couple of places in selftest to
use it. So it doesn't complicate selftest too much.
Socket-wrapper is still very much needed by selftest. However, using
namespaces offers some potential benefits:
- Better testing of DNS. Longer-term we'd like to be able to actually
run BIND9 in selftest.
- More realistic testing of Samba, E.g. we've previously noticed LMDB
was 20% slower when run in a testenv compared to when run outside.
- You can (in theory) join a Windows VM to the samba testenv, or point
the RSAT GUI at a samba testenv.
- You can now have multiple shells that can all run samba utilities
simultaneously.
- The customdc testenv becomes a lot more powerful, e.g. it essentially
becomes a light-weight samba VM that's trivial to spin up.
The basic usage is:
USE_NAMESPACES=1 SELFTEST_TESTENV=ad_dc make testenv
There's then a ./st/<testenv>/nsenter.sh helper script you can run to
attach other shells to the testenv namespace. There's some
example-output attached of using nsenter.sh.
Merge request: https://gitlab.com/samba-team/samba/merge_requests/511
-------------- next part --------------
From f99630f51de7cfe0bc08ee1abd5024ba2b2be41a Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 31 May 2019 12:33:59 +1200
Subject: [PATCH 1/9] selftest: Allow for wider range of terminals (besides
xterm)
Allow developers to override the default @term_args, as well as the
terminal itself.
Currently, due to the nature of the args we pass to xterm (i.e. 'echo -e
"blah.." && bash'), it doesn't make it very flexible for use with other
terminals. By dropping these additional @term_args, it makes it much
easier to slot in an alternative terminal.
For example, these commands now work (more or less).
TERMINAL="terminator" TERMINAL_ARGS="-x bash" \
SELFTEST_TESTENV=ad_dc make testenv
TERMINAL="bash" TERMINAL_ARGS="" \
SELFTEST_TESTENV=nt4_dc make testenv
TERMINAL="bash" TERMINAL_ARGS="--norc" \
SELFTEST_TESTENV=none make testenv
bash is usable, but a little weird because its output is still being
piped. Also bash with ad_dc is a little weird because we're using tee
for the DC's stdout. (I'd also recommend --norc, as it makes it easier
to differentiate between the testenv shell).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/selftest.pl | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/selftest/selftest.pl b/selftest/selftest.pl
index 228bb29..1bbe2f8 100755
--- a/selftest/selftest.pl
+++ b/selftest/selftest.pl
@@ -964,6 +964,10 @@ $envvarstr
my @term = ();
if ($ENV{TERMINAL}) {
@term = ($ENV{TERMINAL});
+ # override the default terminal args (if specified)
+ if (defined($ENV{TERMINAL_ARGS})) {
+ @term_args = split(/ /, $ENV{TERMINAL_ARGS});
+ }
} else {
@term = ("xterm", "-e");
unshift(@term_args, ("bash", "-c"));
--
2.7.4
From 262cd31c4547eb9083bbe0d242154a78a1fec368 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue, 26 Feb 2019 15:54:34 +1300
Subject: [PATCH 2/9] selftest: Add helper scripts to run selftest in
namespaces
This adds the underlying scripts, but they are not actually hooked up to
the selftest code yet, and so are not actually used.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/ns/README | 65 +++++++++++++++++++++++++++++++++++++++++
selftest/ns/add_bridge_iface.sh | 22 ++++++++++++++
selftest/ns/create_bridge.sh | 19 ++++++++++++
selftest/ns/start_in_ns.sh | 56 +++++++++++++++++++++++++++++++++++
4 files changed, 162 insertions(+)
create mode 100644 selftest/ns/README
create mode 100755 selftest/ns/add_bridge_iface.sh
create mode 100755 selftest/ns/create_bridge.sh
create mode 100755 selftest/ns/start_in_ns.sh
diff --git a/selftest/ns/README b/selftest/ns/README
new file mode 100644
index 0000000..e9e9d06
--- /dev/null
+++ b/selftest/ns/README
@@ -0,0 +1,65 @@
+The scripts in this directory are experimental and are used to create testenvs
+in separate linux namespaces. This avoids the need for socket-wrapper.
+
+What are Namespaces
+===================
+Namespaces allow the kernel to segregate its system resources (files, CPU,
+etc), so that different processes only see the set of resources they are
+allowed to use. There are several different types of namespace: network,
+user, process, file, IPC, and so on.
+
+Key points to grasp are:
+* Each type of namespace gets managed separately by the kernel, i.e. process
+namespaces are managed separately to network namespaces, which are separate
+to user namespaces. These scripts give each testenv its own network namespace,
+but otherwise they all still share the same user/process/etc namespace.
+(In future, we may want to give each testenv its own process and user
+namespace, to better mimic a production DC).
+* Namespaces are created using the 'unshare' utility. The new selftest
+namespaces are anonymous/nameless, and so the different namespaces are
+identified by the PID of the processes running within the namespace
+(typically samba).
+* Linux supports nesting namespaces within namespaces. In this case, each
+testenv DC has its own network namespace, which is a child of the overarching
+selftest namespace (which itself is a child of whatever namespace you run
+'make test' from - usually this would be the root namespace).
+
+How does it work?
+=================
+Normally when 'make test' is run, every testenv uses a 127.0.0.x IP address
+and socket-wrapper passes the packets between them.
+
+With namespaces, we can use real IP addresses and have the packets pass through
+the kernel's IP stack normally, as it forwards them between namespaces.
+
+We use veth interfaces for this. veth is a type of virtual interface supported
+by the kernel. veth interfaces come in pairs, and act as a tunnel - any packets
+sent on a veth interface simply end up as received packets on the pair veth
+interface.
+
+We create a new veth interface pair for each testenv, and use them to connect
+up the namespaces. One end of the veth pair is added to the main selftest
+namespace, and the other end is added to a new namespace that we'll run
+samba in. E.g.
+
+selftest.pl veth21-br ------------------------ veth21 samba (ad_dc_ntvfs)
+ 10.0.0.11 10.0.0.21
+ Namespace 1 Namespace 2
+
+However, we need to run multiple different testenvs and have them talk to
+each other. So to do this, we need a bridge interface ('selftest0') to connect
+up the namespaces, which essentially just acts as a hub. So connecting together
+multiple testenvs looks more like this:
+
+selftest.pl +-- veth21-br ------------------------ veth21 samba (ad_dc_ntvfs)
+ | 10.0.0.21
+ selftest0 --+ Namespace 2
+ 10.0.0.11 |
+ +-- veth22-br ------------------------ veth22 samba (vampire_dc)
+ 10.0.0.22
+ Namespace 1 Namespace 3
+
+The veth interfaces are named vethX and vethX-br, where X is the
+SOCKET_WRAPPER_DEFAULT_IFACE for the testenv. The vethX-br interface is always
+added to the selftest0 bridge interface.
+
diff --git a/selftest/ns/add_bridge_iface.sh b/selftest/ns/add_bridge_iface.sh
new file mode 100755
index 0000000..da9d53a
--- /dev/null
+++ b/selftest/ns/add_bridge_iface.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+#
+# Configures the interfaces needed for communication between namespaces.
+# This handles the bridge-end of the veth pair.
+interface=$1
+
+# the main bridge interface is called 'selftest0' (although in future we may
+# want to segregate the different domains by using different bridges)
+bridge=$2
+
+# we need to wait for the child namespace to start up and add the new
+# interface back to our new namespace
+while ! ip link show $interface > /dev/null 2>&1
+do
+ sleep 0.1
+ echo "Waiting for $interface to be created..."
+done
+
+# bring the bridge-end of the link up and add it to the bridge
+ip link set dev $interface up
+ip link set $interface master $bridge
+
diff --git a/selftest/ns/create_bridge.sh b/selftest/ns/create_bridge.sh
new file mode 100755
index 0000000..9766cd8
--- /dev/null
+++ b/selftest/ns/create_bridge.sh
@@ -0,0 +1,19 @@
+#!/bin/sh
+
+# creates a bridge interface (i.e. 'selftest0') that connects together the
+# veth interfaces for the various testenvs
+
+br_name=$1
+ip_addr=$2
+ipv6_addr=$3
+
+# make sure the loopback is up (needed for pinging between namespaces, etc)
+ip link set dev lo up
+
+# create the bridge interface and enable it
+ip link add $br_name type bridge
+ip addr add $ip_addr/24 dev $br_name
+ip addr add $ipv6_addr/112 dev $br_name
+ip link set $br_name up
+
+
diff --git a/selftest/ns/start_in_ns.sh b/selftest/ns/start_in_ns.sh
new file mode 100755
index 0000000..5831a0b
--- /dev/null
+++ b/selftest/ns/start_in_ns.sh
@@ -0,0 +1,56 @@
+#!/bin/sh
+#
+# Starts samba in a separate namespace. This gets passed the interface/IP
+# to use, as well as the Samba command to run. The whole script gets run
+# (via unshare) in a separate namespace.
+
+# the first 3 args are our interface-name, parent-PID, and a exports file
+# containing environment variables ($SERVER, $SERVER_IP, etc)
+interface=$1
+exports_file=$2
+parent_pid=$3
+
+# we write the testenv environment variables to file, which makes it easier
+# to work out the $SERVER, $SERVER_IP, etc
+. $exports_file
+
+# The namespaces we use are anonymous, which means other processes would need
+# to use our PID to access the new namespace
+echo "-------------------------------------------------------------"
+echo "Created namespace for $NETBIOSNAME"
+echo "To communicate with this testenv, use: nsenter -t $$ --net sh"
+echo "To copy its environment variables, use: . $exports_file"
+echo "-------------------------------------------------------------"
+
+# the rest of the args are the samba command to run
+shift 3
+SAMBA_CMD=$@
+
+# make sure namespace loopback is up (it's needed for ping, etc)
+ip link set dev lo up
+
+# Create the interfaces needed for communication between namespaces.
+# We use a veth pair, which acts as a tunnel between the namespaces.
+# One end of the veth link is added to a common bridge in the top-level (i.e.
+# selftest) namespace, and the other end is added to the testenv's namespace.
+# This means each testenv DC is in its own namespace, but they can talk to
+# each other via the common bridge interface.
+# The new veth interfaces are named "vethX" and "vethX-br", where
+# X = the testenv IP (i.e. Samba::get_interface()). E.g. ad_dc = veth30,
+# and veth30-br.
+# The "vethX" interface will live in the new testenv's namespace.
+# The "vethX-br" end is added to the bridge in the main selftest namespace.
+ip link add dev $interface-br type veth peer name $interface
+
+# move the bridge end of the link back into the parent namespace.
+ip link set $interface-br netns $parent_pid
+
+# configure our IP address and bring the interface up
+ip addr add $SERVER_IP/24 dev $interface
+# Note that samba can't bind to the IPv6 address while DAD is in progress,
+# so we use 'nodad' when configuring the address
+ip addr add $SERVER_IPV6/112 dev $interface nodad
+ip link set dev $interface up
+
+# start samba
+$SAMBA_CMD
--
2.7.4
From 649571925eb88b0fa61c4d0cba37e254d87a89d2 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 14 Mar 2019 17:38:22 +1300
Subject: [PATCH 3/9] selftest: Add TESTENV_DIR "env" variable
We store the testenv directory path for the 'ctx' hashmap, but not for
the testenv-vars hashmap (and that can be really annoying sometimes).
Add it into the second hashmap that selftest actually keeps track of.
Currently it's only stored in the hashmap, not actually exported as an
environment variable (but we could easily do that if a test-case need
this info).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/target/Samba3.pm | 1 +
selftest/target/Samba4.pm | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/selftest/target/Samba3.pm b/selftest/target/Samba3.pm
index 5e28253..9886395 100755
--- a/selftest/target/Samba3.pm
+++ b/selftest/target/Samba3.pm
@@ -2387,6 +2387,7 @@ force_user:x:$gid_force_user:
$ret{SMBD_TEST_LOG} = "$prefix/smbd_test.log";
$ret{SMBD_TEST_LOG_POS} = 0;
$ret{SERVERCONFFILE} = $conffile;
+ $ret{TESTENV_DIR} = $prefix_abs;
$ret{CONFIGURATION} ="-s $conffile";
$ret{LOCK_DIR} = $lockdir;
$ret{SERVER} = $server;
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 56ca8bd..609ff83 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -376,8 +376,8 @@ sub setup_dns_hub_internal($$$)
$env->{SERVER_IPV6} = Samba::get_ipv6_addr($hostname);
$env->{SOCKET_WRAPPER_DEFAULT_IFACE} = Samba::get_interface($hostname);
$env->{DNS_HUB_LOG} = "$prefix_abs/dns_hub.log";
-
$env->{RESOLV_CONF} = "$prefix_abs/resolv.conf";
+ $env->{TESTENV_DIR} = $prefix_abs;
open(RESOLV_CONF, ">$env->{RESOLV_CONF}");
print RESOLV_CONF "nameserver $env->{SERVER_IP}\n";
@@ -916,6 +916,7 @@ nogroup:x:65534:nobody
PRIVATEDIR => $ctx->{privatedir},
BINDDNSDIR => $ctx->{binddnsdir},
SERVERCONFFILE => $ctx->{smb_conf},
+ TESTENV_DIR => $ctx->{prefix_abs},
CONFIGURATION => $configuration,
SOCKET_WRAPPER_DEFAULT_IFACE => $ctx->{swiface},
NSS_WRAPPER_PASSWD => $ctx->{nsswrap_passwd},
--
2.7.4
From 58c01e87941028855689d5d7049bc4a7f82bed44 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 23 May 2019 17:44:37 +1200
Subject: [PATCH 4/9] selftest: Add linux namespace support (USE_NAMESPACES=1)
This hooks up the selftest/ns/* scripts added earlier with the selftest
system, so developers can optionally run a testenv or test using linux
namespaces instead of socket-wrapper.
The idea is this is experimental functionality that we can extend
further in future, in order to make testing Samba more versatile.
+ The top-level WAF script now does an 'unshare' to create a new
top-level 'selftest' namespace in which to create the testenv(s).
+ selftest.pl creates a common 'selftest0' bridge to connect together
the individual DCs.
+ Update Samba.pm so it can use real IPs instead of loopback addresses.
In fork_and_exec(), we add a couple of hooks so that the binary gets
started in a different namespace (using unshare/start_in_ns.sh), and
the parent process connects the new child namespace up to the common
selftest0 bridge (using add_bridge_iface.sh).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/selftest.pl | 10 +++++
selftest/target/Samba.pm | 109 ++++++++++++++++++++++++++++++++++++++++++++--
selftest/target/Samba4.pm | 3 ++
selftest/wscript | 12 ++++-
4 files changed, 130 insertions(+), 4 deletions(-)
diff --git a/selftest/selftest.pl b/selftest/selftest.pl
index 1bbe2f8..9e3d818 100755
--- a/selftest/selftest.pl
+++ b/selftest/selftest.pl
@@ -424,6 +424,16 @@ if ($opt_libuid_wrapper_so_path) {
}
}
+if (defined($ENV{USE_NAMESPACES})) {
+ print "Using linux containerization for selftest testenv(s)...\n";
+
+ # Create a common bridge to connect up the testenv namespaces. We give
+ # it the client's IP address, as this is where the tests will run from
+ my $ipv4_addr = Samba::get_ipv4_addr("client");
+ my $ipv6_addr = Samba::get_ipv6_addr("client");
+ system "$ENV{SRCDIR_ABS}/selftest/ns/create_bridge.sh selftest0 $ipv4_addr $ipv6_addr";
+}
+
$ENV{LD_PRELOAD} = $ld_preload;
print "LD_PRELOAD=$ENV{LD_PRELOAD}\n";
diff --git a/selftest/target/Samba.pm b/selftest/target/Samba.pm
index 6fd8d01..ca3099c 100644
--- a/selftest/target/Samba.pm
+++ b/selftest/target/Samba.pm
@@ -516,7 +516,13 @@ sub get_ipv4_addr
$swiface += $iface_num;
}
- return "127.0.0.$swiface";
+ if (use_namespaces()) {
+ # use real IPs if selftest is running in its own network namespace
+ return "10.0.0.$swiface";
+ } else {
+ # use loopback IPs with socket-wrapper
+ return "127.0.0.$swiface";
+ }
}
sub get_ipv6_addr
@@ -541,7 +547,12 @@ sub get_interfaces_config
}
for (my $i = 0; $i < $num_ips; $i++) {
my $ipv4_addr = Samba::get_ipv4_addr($hostname, $i);
- $interfaces .= "$ipv4_addr/8 ";
+ if (use_namespaces()) {
+ # use a /24 subnet with network namespaces
+ $interfaces .= "$ipv4_addr/24 ";
+ } else {
+ $interfaces .= "$ipv4_addr/8 ";
+ }
}
my $ipv6_addr = Samba::get_ipv6_addr($hostname);
@@ -627,10 +638,13 @@ sub fork_and_exec
unlink($daemon_ctx->{LOG_FILE});
print "STARTING $daemon_ctx->{NAME} for $ENV{ENVNAME}...";
+ my $parent_pid = $$;
my $pid = fork();
# exec the daemon in the child process
if ($pid == 0) {
+ my @preargs = ();
+
# redirect the daemon's stdout/stderr to a log file
if (defined($daemon_ctx->{TEE_STDOUT})) {
# in some cases, we want out from samba to go to the log file,
@@ -671,12 +685,27 @@ sub fork_and_exec
close($env_vars->{STDIN_PIPE});
open STDIN, ">&", $STDIN_READER or die "can't dup STDIN_READER to STDIN: $!";
+ # if using kernel namespaces, prepend the command so the process runs in
+ # its own namespace
+ if (Samba::use_namespaces()) {
+ @preargs = ns_exec_preargs($parent_pid, $env_vars);
+ }
+
# the command args are stored as an array reference (because...Perl),
# so convert the reference back to an array
my @full_cmd = @{ $daemon_ctx->{FULL_CMD} };
- exec(@full_cmd) or die("Unable to start $ENV{MAKE_TEST_BINARY}: $!");
+
+ exec(@preargs, @full_cmd) or die("Unable to start $ENV{MAKE_TEST_BINARY}: $!");
}
+
print "DONE ($pid)\n";
+
+ # if using kernel namespaces, we now establish a connection between the
+ # main selftest namespace (i.e. this process) and the new child namespace
+ if (use_namespaces()) {
+ ns_child_forked($pid, $env_vars);
+ }
+
return $pid;
}
@@ -797,4 +826,78 @@ sub export_envvars_to_file
close(FILE);
}
+# Returns true if kernel namespaces are being used instead of socket-wrapper.
+# The default is false.
+sub use_namespaces
+{
+ return defined($ENV{USE_NAMESPACES});
+}
+
+# returns a given testenv's interface-name (only when USE_NAMESPACES=1)
+sub ns_interface_name
+{
+ my ($hostname) = @_;
+
+ # when using namespaces, each testenv has its own vethX interface,
+ # where X = Samba::get_interface(testenv_name)
+ my $iface = get_interface($hostname);
+ return "veth$iface";
+}
+
+# Called after a new child namespace has been forked
+sub ns_child_forked
+{
+ my ($child_pid, $env_vars) = @_;
+
+ # we only need to do this for the first child forked for this testenv
+ if (defined($env_vars->{NS_PID})) {
+ return;
+ }
+
+ # store the child PID. It's the only way the main (selftest) namespace can
+ # access the new child (testenv) namespace.
+ $env_vars->{NS_PID} = $child_pid;
+
+ # Add the new child namespace's interface to the main selftest bridge.
+ # This connects together the various testenvs so that selftest can talk to
+ # them all
+ my $iface = ns_interface_name($env_vars->{NETBIOSNAME});
+ system "$ENV{SRCDIR}/selftest/ns/add_bridge_iface.sh $iface-br selftest0";
+}
+
+# returns args to prepend to a command in order to execute it the correct
+# namespace for the testenv (creating a new namespace if needed).
+# This should only used when USE_NAMESPACES=1 is set.
+sub ns_exec_preargs
+{
+ my ($parent_pid, $env_vars) = @_;
+
+ # NS_PID stores the pid of the first child daemon run in this namespace
+ if (defined($env_vars->{NS_PID})) {
+
+ # the namespace has already been created previously. So we use nsenter
+ # to execute the command in the given testenv's namespace. We need to
+ # use the NS_PID to identify this particular namespace
+ return ("nsenter", "-t", "$env_vars->{NS_PID}", "--net");
+ } else {
+
+ # We need to create a new namespace for this daemon (i.e. we're
+ # setting up a new testenv). First, write the environment variables to
+ # an exports.sh file for this testenv (for convenient access by the
+ # namespace scripts).
+ my $exports_file = "$env_vars->{TESTENV_DIR}/exports.sh";
+ export_envvars_to_file($exports_file, $env_vars);
+
+ # when using namespaces, each testenv has its own veth interface
+ my $interface = ns_interface_name($env_vars->{NETBIOSNAME});
+
+ # we use unshare to create a new network namespace. The start_in_ns.sh
+ # helper script gets run first to setup the new namespace's interfaces.
+ # (This all gets prepended around the actual command to run in the new
+ # namespace)
+ return ("unshare", "--net", "$ENV{SRCDIR}/selftest/ns/start_in_ns.sh",
+ $interface, $exports_file, $parent_pid);
+ }
+}
+
1;
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 609ff83..b1c6aa4 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -468,6 +468,9 @@ sub get_cmd_env_vars
return $cmd_env;
}
+# Sets up a forest trust namespace.
+# (Note this is different to kernel namespaces, setup by the
+# USE_NAMESPACES=1 option)
sub setup_namespaces($$:$$)
{
my ($self, $localenv, $upn_array, $spn_array) = @_;
diff --git a/selftest/wscript b/selftest/wscript
index 5c864eb..f204f34 100644
--- a/selftest/wscript
+++ b/selftest/wscript
@@ -246,9 +246,12 @@ def cmd_testonly(opt):
env.OPTIONS += " --nss_wrapper_so_path=" + CONFIG_GET(opt, 'LIBNSS_WRAPPER_SO_PATH')
env.OPTIONS += " --resolv_wrapper_so_path=" + CONFIG_GET(opt, 'LIBRESOLV_WRAPPER_SO_PATH')
- env.OPTIONS += " --socket_wrapper_so_path=" + CONFIG_GET(opt, 'LIBSOCKET_WRAPPER_SO_PATH')
env.OPTIONS += " --uid_wrapper_so_path=" + CONFIG_GET(opt, 'LIBUID_WRAPPER_SO_PATH')
+ # selftest can optionally use kernel namespaces instead of socket-wrapper
+ if os.environ.get('USE_NAMESPACES') is None:
+ env.OPTIONS += " --socket_wrapper_so_path=" + CONFIG_GET(opt, 'LIBSOCKET_WRAPPER_SO_PATH')
+
#if unversioned_sys_platform in ('freebsd', 'netbsd', 'openbsd', 'sunos'):
# env.OPTIONS += " --use-dns-faking"
@@ -277,6 +280,13 @@ def cmd_testonly(opt):
# We use the full path rather than relative path to avoid problems on some platforms (ie. solaris 8).
env.CORE_COMMAND = '${PERL} ${srcdir}/selftest/selftest.pl --target=${SELFTEST_TARGET} --prefix=${SELFTEST_PREFIX} --srcdir=${srcdir} --exclude=${srcdir}/selftest/skip ${TESTLISTS} ${OPTIONS} ${TESTS}'
+ # If using namespaces (rather than socket-wrapper), run the selftest script
+ # in its own network namespace (by doing an 'unshare'). (To create a new
+ # namespace as a non-root user, we have to also unshare the current user
+ # namespace, and remap ourself as root in the namespace created)
+ if os.environ.get('USE_NAMESPACES') is not None:
+ env.CORE_COMMAND = 'unshare --net --user --map-root-user ' + env.CORE_COMMAND
+
if env.ADDRESS_SANITIZER:
# For now we cannot run with leak detection
no_leak_check = "ASAN_OPTIONS=detect_leaks=0"
--
2.7.4
From 9e8f0153017d85b8815914d79cc232d2a04a95dd Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 28 Mar 2019 17:40:46 +1300
Subject: [PATCH 5/9] selftest: Add helper scripts for accessing the testenv
namespace
This patch adds some helper scripts that make talking to a given
testenv's namespace slightly easier.
One of the really cool things about namespaces is you can run multiple
different programs that can all talk to the testenv DC. However, the
command to do this is a bit unweildly, it's based on PID so it changes
everytime you start up a testenv, and you loose all the environment
variables that selftest normally sets up.
This patch adds a couple of helper scripts:
- nsenter-helper.sh: this takes the variables defined in an exports_file
and exports them all. It prints some basic help and then starts a new
shell session (this whole script gets run in the new namespace).
Essentially this achieves something similar to the legacy
selftest-vars.sh script (except this one actually works).
- mk_nsenter.sh: this generates a simple wrapper script that'll run
nsenter and then call nsenter-helper.sh. A separate wrapper script
gets created for each testenv. E.g. to run it, just go:
./st/ad_dc/nsenter.sh
This is a wrapper for a more complicated command underneath like:
nsenter -t 437353 --net --user --preserve-credentials \
/home/timbeale/code/samba/selftest/ns/nsenter-helper.sh \
/home/timbeale/code/samba/st/ad_dc/exports.sh
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/ns/mk_nsenter.sh | 32 ++++++++++++++++++++++++++++++++
selftest/ns/nsenter-helper.sh | 31 +++++++++++++++++++++++++++++++
selftest/ns/start_in_ns.sh | 11 ++++++++---
3 files changed, 71 insertions(+), 3 deletions(-)
create mode 100755 selftest/ns/mk_nsenter.sh
create mode 100755 selftest/ns/nsenter-helper.sh
diff --git a/selftest/ns/mk_nsenter.sh b/selftest/ns/mk_nsenter.sh
new file mode 100755
index 0000000..f175d6b
--- /dev/null
+++ b/selftest/ns/mk_nsenter.sh
@@ -0,0 +1,32 @@
+#!/bin/sh
+#
+# Helper script. If you want a 2nd shell that communicates with the testenv DC
+# you can use the nsenter command to change the namespace you're in. However,
+# this command is a bit unwieldly and changes depending on the testenv PID.
+# We can generate a helper script on the fly that abstracts all this
+# complexity, allowing you to use the same, simple command to change the
+# namespace that you're in, e.g.
+# st/ad_dc/nsenter.sh
+
+pid=$1
+exports_file=$2
+
+# The basic command to enter the testenv's network namespace.
+# We enter the user namespace as well (as ourself, which is really the root
+# user for the namespace), otherwise we need sudo to make this work.
+nsenter_cmd="nsenter -t $pid --net --user --preserve-credentials"
+
+# By default, the nsenter command will just start a new shell in the namespace.
+# we use a wrapper helper script, which first loads all the environment
+# variables that are usually defined in selftest (and prints some basic help).
+helper_script="$(dirname $0)/nsenter-helper.sh $exports_file"
+
+# generate the dynamic script
+dyn_script="$(dirname $2)/nsenter.sh"
+echo "#!/bin/sh" > $dyn_script
+echo "$nsenter_cmd $helper_script" >> $dyn_script
+chmod 755 $dyn_script
+
+# return the script we created
+echo "$dyn_script"
+
diff --git a/selftest/ns/nsenter-helper.sh b/selftest/ns/nsenter-helper.sh
new file mode 100755
index 0000000..f396ed4
--- /dev/null
+++ b/selftest/ns/nsenter-helper.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# Helper script that gets run with nsenter to manually setup a secondary shell
+# session to a given namespace testenv. This basically just sets up the same
+# environment variables as you normally get with selftest, for convenience.
+
+if [ $# -lt 1 ] ; then
+ echo "Usage: $0 <exports-file>"
+ exit 1
+fi
+
+# we get passed a exports file with all the environment variables defined
+exports_file=$1
+
+# read the exports file so the new shell has appropriate variables setup
+# (we export rather than sourcing here so they get inherited by the subshell)
+while read -r line ; do
+ export $line
+ # dump them for the user too
+ echo $line
+done < $exports_file
+
+echo ""
+echo "Entered $NETBIOSNAME namespace, with above variables defined."
+echo "Use CTRL+D or exit to leave the namespace."
+echo ""
+
+# start a shell session in the new namespace
+$SHELL
+
+
diff --git a/selftest/ns/start_in_ns.sh b/selftest/ns/start_in_ns.sh
index 5831a0b..f16767d 100755
--- a/selftest/ns/start_in_ns.sh
+++ b/selftest/ns/start_in_ns.sh
@@ -17,9 +17,14 @@ parent_pid=$3
# The namespaces we use are anonymous, which means other processes would need
# to use our PID to access the new namespace
echo "-------------------------------------------------------------"
-echo "Created namespace for $NETBIOSNAME"
-echo "To communicate with this testenv, use: nsenter -t $$ --net sh"
-echo "To copy its environment variables, use: . $exports_file"
+echo "Created namespace for $NETBIOSNAME ($ENVNAME) PID $$"
+
+# generate a helper script if the developer wants to talk to this namespace
+# in another shell
+mk_nsenter_script="$(dirname $0)/mk_nsenter.sh"
+helper_script=$($mk_nsenter_script $$ $exports_file)
+
+echo "To communicate with this testenv, use: $helper_script"
echo "-------------------------------------------------------------"
# the rest of the args are the samba command to run
--
2.7.4
From 45be6471c48371d0cf75002e0557d67e66085957 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 31 May 2019 11:23:49 +1200
Subject: [PATCH 6/9] selftest: Add more notes on using selftest with
namespaces
In particular, document how to hook up a testenv to a Windows VM
(ideally there should be a helper script to do this, but in the
meantime some instructions are better than nothing).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/ns/README | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
diff --git a/selftest/ns/README b/selftest/ns/README
index e9e9d06..a8ad1c0 100644
--- a/selftest/ns/README
+++ b/selftest/ns/README
@@ -63,3 +63,100 @@ The veth interfaces are named vethX and vethX-br, where X is the
SOCKET_WRAPPER_DEFAULT_IFACE for the testenv. The vethX-br interface is always
added to the selftest0 bridge interface.
+How do I use it?
+================
+To use namespaces instead of socket-wrapper, just add 'USE_NAMESPACES=1' to the
+make command, e.g.
+
+To run the 'quick' test cases using namespaces:
+USE_NAMESPACES=1 make test TESTS=quick
+
+To setup an ad_dc testenv using namespaces:
+USE_NAMESPACES=1 SELFTEST_TESTENV=ad_dc make testenv
+
+You can connect secondary shells to the namespace your testenv is running in.
+The command to do this is a little complicated, so a helper 'nsenter.sh' script
+gets autogenerated when the testenv is created. E.g. to connect to the testenv
+that the ad_dc is running in, use:
+./st/ad_dc/nsenter.sh
+
+This script also sets up the shell with all the same $SERVER/$USERNAME/etc
+variables that you normally get in xterm.
+
+To run the ad-dc-backup autobuild job using namespaces:
+USE_NAMESPACES=1 script/autobuild.py samba-ad-dc-backup --verbose --nocleanup \
+ --keeplogs --tail --testbase /tmp/samba-testbase
+
+Using the customdc testenv, you can basically now essentially your own
+light-weight samba VM. E.g.
+MY_BACKUP=/home/$USER/samba-backup-prod-domain.tar.bz2
+USE_NAMESPACES=1 BACKUP_FILE=$MY_BACKUP SELFTEST_TESTENV=customdc make testenv
+
+You can then talk to that DC in any other shell by using
+./st/customdc/nsenter.sh which enters the DC's network namespace (with
+all the $SERVER/etc env variables defined).
+
+How to join VMs to the testenv
+----------------------------------------
+I haven't tried this (beyond basic IP connectivity), but using namespaces it
+should now be possible to connect a Windows VM to a Samba testenv.
+
+1. Work out the main selftest.pl namespace PID manually, e.g.
+SELFTEST_PID= ps waux | grep selftest.pl
+
+2. Create a new veth to bridge between the selftest namespace and your PC's
+default namespace:
+sudo ip link add dev testenv-veth0 type veth peer name testenv-veth1
+
+3. Move one end of the veth tunnel into the selftest namespace:
+sudo ip link set testenv-veth1 netns $SELFTEST_PID
+
+4. Configure the veth end in the default namespace to be in the same subnet
+as the selftest network:
+sudo ip link set dev testenv-veth0 up
+sudo ip addr add 10.0.0.63/24 dev testenv-veth0
+
+5. Enter the selftest namespace, bring that end of the pipe up, and add it to
+to the main selftest0 bridge (that connects all the DCs together). We also need
+to add a default route from selftest back to your PC's default namespace.
+nsenter -t $SELFTEST_PID --net --user --preserve-credentials
+ip link set dev testenv-veth1 up
+ip link set testenv-veth1 master selftest0
+ip route add default via 10.0.0.63
+logout
+
+Your Windows VM and samba testenv should now be able to talk to each
+other over IP!
+
+6. The other step is to get DNS working. You probably need to add dns_hub
+(10.0.0.64) as a nameserver (at least on your Windows VM).
+
+This should work for using RSAT tools on samba, or joining Windows to Samba
+(depending on the schema version). Joining samba to Windows is a bit more
+tricky, as the namespaces are tied to the *running* samba process.
+
+What you'd probably want to do is run the join command to the windows VM
+outside of testenv, create an offline backup-file of the resulting DB, and
+then plug that backup-file into the customdc testenv. (And then follow the
+above veth/bridge steps to join samba to the VM).
+
+Note that the namespace disappears once you stop the testenv, so you'd
+need to do the above steps with creating the veth interface every time
+you restarted the testenv.
+
+Known limitations
+=================
+- When running a testenv, sometimes xterm can fail to startup, due to a
+ permissions problem with /dev/pts. This seems to be a particular problem
+ with the 'none' testenv.
+ A short-term work-around is to use a terminal that doesn't try to access
+ /dev/pts, e.g. just use bash as the terminal:
+ TERMINAL=bash TERMINAL_ARGS='--norc' USE_NAMESPACES=1 \
+ SELFTEST_TESTENV=none make testenv
+- Some test cases rely on socket-wrapper, so will fail when run using
+ namespaces.
+- Currently USE_NAMESPACES maps you (i.e. $USER) to root in the new namespace.
+ This means any test cases that rely on being a non-root user will fail (i.e.
+ anything that fails under 'sudo make test' will also fail with namespaces).
+- Namespaces should work within docker, but currently the 'unshare' system
+ call is disallowed on the gitlab CI runners.
--
2.7.4
From c9e8dff5661caf7e8ca6744f4ca4fba4d88e03d6 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Wed, 3 Oct 2018 08:56:45 +1300
Subject: [PATCH 7/9] provision: Fallback to assumption root-UID==zero
Which is not a terrible assumption to make. The super-user on linux will
always have UID of zero, however, the super-user will not necessarily be
called "root".
This makes the provision/join commands work better when run in a
container. (And while deploying Samba in a container is perhaps not the
smartest move, this gives us some versatility when testing Samba).
This is needed to get the provision commands working in the domain_backup
tests when run with USE_NAMESPACES=1.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
python/samba/provision/__init__.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/python/samba/provision/__init__.py b/python/samba/provision/__init__.py
index 2bb2614..14ab41b 100644
--- a/python/samba/provision/__init__.py
+++ b/python/samba/provision/__init__.py
@@ -540,6 +540,16 @@ def findnss_gid(names):
return findnss(grp.getgrnam, names)[2]
+def get_root_uid(root, logger):
+ try:
+ root_uid = findnss_uid(root)
+ except KeyError as e:
+ logger.info(e)
+ logger.info("Assuming root user has UID zero")
+ root_uid = 0
+ return root_uid
+
+
def provision_paths_from_lp(lp, dnsdomain):
"""Set the default paths for provisioning.
@@ -2152,7 +2162,7 @@ def provision(logger, session_info, smbconf=None,
if domainsid is None:
domainsid = security.random_sid()
- root_uid = findnss_uid([root or "root"])
+ root_uid = get_root_uid([root or "root"], logger)
nobody_uid = findnss_uid([nobody or "nobody"])
users_gid = findnss_gid([users or "users", 'users', 'other', 'staff'])
root_gid = pwd.getpwuid(root_uid).pw_gid
--
2.7.4
From 8b66c14d716404da8acc1a4071d6977e43f91656 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 30 May 2019 14:46:35 +1200
Subject: [PATCH 8/9] selftest: Add check customdc has valid realm/domain
If we couldn't determine the realm/domain from the backup file, it's a
lot nicer to fail early with a clear error message (rather than failing
later on with a really obscure message).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/target/Samba4.pm | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index b1c6aa4..d647a5c 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -3310,6 +3310,10 @@ sub setup_customdc
# work out the correct domain/realm env values from the backup-file
my ($domain, $realm) = $self->get_backup_domain_realm($backup_file);
+ if ($domain eq '' or $realm eq '') {
+ warn("Could not determine domain or realm");
+ return undef;
+ }
# create a placeholder directory and smb.conf, as well as the env vars.
my ($env, $ctx) = $self->prepare_dc_testenv($prefix, $dc_name,
--
2.7.4
From 9330b481e2e5e0a1bcf42c1351d75cc5a1e0bb8c Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 30 May 2019 14:55:52 +1200
Subject: [PATCH 9/9] selftest: Don't use global dirs when parsing customdc
realm
When creating the customdc, testparm would default to using
/usr/local/samba sub-directories for creating sockets and lock files.
Instead, pass in the tmpdir we just created as an option to the command.
Normally this didn't cause a noticeable problem, however, if we run the
command with UID-wrapper but without socket-wrapper (i.e.
USE_NAMESPACES=1), then it fails completely.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/target/Samba4.pm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index d647a5c..c661a10 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -3280,10 +3280,15 @@ sub get_backup_domain_realm
return undef, undef;
}
+ # make sure we don't try to create locks/sockets in the default install
+ # location (i.e. /usr/local/samba/)
+ my $options = "--option=\"private dir = $tmpdir\"";
+ $options .= " --option=\"lock dir = $tmpdir\"";
+
# now use testparm to read the values we're interested in
my $testparm = Samba::bindir_path($self, "testparm");
- my $domain = `$testparm $smbconf -sl --parameter-name=WORKGROUP`;
- my $realm = `$testparm $smbconf -sl --parameter-name=REALM`;
+ my $domain = `$testparm $smbconf -sl --parameter-name=WORKGROUP $options`;
+ my $realm = `$testparm $smbconf -sl --parameter-name=REALM $options`;
chomp $realm;
chomp $domain;
print "Backup-file REALM is $realm, DOMAIN is $domain\n";
--
2.7.4
-------------- next part --------------
timbeale at timbeale-pc:~/code/samba$ ./st/ad_dc/nsenter.sh
DOMAIN=ADDOMAIN
DNSNAME=addom.samba.example.com
REALM=ADDOM.SAMBA.EXAMPLE.COM
DOMSID=S-1-5-21-508928020-2655384664-4115864588
DC_SERVER=addc
DC_SERVER_IP=10.0.0.30
DC_SERVER_IPV6=fd00:0000:0000:0000:0000:0000:5357:5f1e
DC_NETBIOSNAME=ADDC
SERVER=addc
SERVER_IP=10.0.0.30
SERVER_IPV6=fd00:0000:0000:0000:0000:0000:5357:5f1e
NETBIOSNAME=ADDC
SAMSID=S-1-5-21-508928020-2655384664-4115864588
SERVERCONFFILE=/home/timbeale/code/samba/st/ad_dc/etc/smb.conf
USERNAME=Administrator
PASSWORD=locDCpass1
DC_USERNAME=Administrator
DC_PASSWORD=locDCpass1
UID_RFC2307TEST=65533
GID_RFC2307TEST=65532
KRB5_CONFIG=/home/timbeale/code/samba/st/ad_dc/etc/krb5.conf
SELFTEST_WINBINDD_SOCKET_DIR=/home/timbeale/code/samba/st/ad_dc/winbindd_socket
LOCAL_PATH=/home/timbeale/code/samba/st/ad_dc/share
RESOLV_CONF=/home/timbeale/code/samba/st/dns_hub/rootdnsforwarder/resolv.conf
NSS_WRAPPER_PASSWD=/home/timbeale/code/samba/st/ad_dc/etc/passwd
NSS_WRAPPER_GROUP=/home/timbeale/code/samba/st/ad_dc/etc/group
NSS_WRAPPER_HOSTS=/home/timbeale/code/samba/st/hosts
NSS_WRAPPER_HOSTNAME=addc.addom.samba.example.com
NSS_WRAPPER_MODULE_SO_PATH=/home/timbeale/code/samba/bin/default/nsswitch/libnss-wrapper-winbind.so
NSS_WRAPPER_MODULE_FN_PREFIX=winbind
RESOLV_WRAPPER_HOSTS=/home/timbeale/code/samba/st/dns_host_file
Entered ADDC namespace, with above variables defined.
Use CTRL+D or exit to leave the namespace.
root at timbeale-pc:~/code/samba# ip -4 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: veth30 at if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link-netnsid 0
inet 10.0.0.30/24 scope global veth30
valid_lft forever preferred_lft forever
root at timbeale-pc:~/code/samba# bin/ldbsearch -H ldap://$SERVER -U$USERNAME%$PASSWORD '(cn=Administrator)' dn
# record 1
dn: CN=Administrator,CN=Users,DC=addom,DC=samba,DC=example,DC=com
# Referral
ref: ldap://addom.samba.example.com/CN=Configuration,DC=addom,DC=samba,DC=example,DC=com
# Referral
ref: ldap://addom.samba.example.com/DC=DomainDnsZones,DC=addom,DC=samba,DC=example,DC=com
# Referral
ref: ldap://addom.samba.example.com/DC=ForestDnsZones,DC=addom,DC=samba,DC=example,DC=com
# returned 4 records
# 1 entries
# 3 referrals
root at timbeale-pc:~/code/samba#
More information about the samba-technical
mailing list