[PATCH] Create a custom testenv from backup-file for manual testing

Tim Beale timbeale at catalyst.net.nz
Tue Oct 30 20:52:19 UTC 2018


The attached patch allows you to easily spin up any custom testenv that
you want, for manual testing. All you need is a backup-file that the
testenv DC will be based off.

The main use-case is any testing that involves a large database.
Creating user accounts is slow. Even with the improvements to the
traffic_replay user generation, creating 5,000 users still takes close
to 10 minutes. Instead of creating a blank testenv and slowly populating
it with users, you can do this step once, take a backup, and then spin
up the backup database multiple times as a new testenv.

This testenv might be useful in other situations too. E.g. testing
migrating databases across samba versions, you spot a DB corruption and
want to try diagnosing/fixing it without completely losing the original
problem state, you want to create a labdc-replica testenv of a real
network, etc.

Also fixed a couple of minor dnsupdate problems that were making
debugging testenv bootstrapping issues difficult.

CI pass: https://gitlab.com/catalyst-samba/samba/pipelines/34756736

-------------- next part --------------
From e4f6ee4680637e14d224d3774193ef1f4b4ecdd9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 26 Oct 2018 11:08:54 +1300
Subject: [PATCH 1/3] selftest: Add new customdc testenv that can load any
 backup-file

This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.

The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.

In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.

To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
  BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
    SELFTEST_TESTENV=customdc make testenv

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
 selftest/target/Samba.pm  |  1 +
 selftest/target/Samba4.pm | 69 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/selftest/target/Samba.pm b/selftest/target/Samba.pm
index a3be713..aa6ec9e 100644
--- a/selftest/target/Samba.pm
+++ b/selftest/target/Samba.pm
@@ -423,6 +423,7 @@ sub get_interface($)
     $interfaces{"renamedc"} = 42;
     $interfaces{"labdc"} = 43;
     $interfaces{"offlinebackupdc"} = 44;
+    $interfaces{"customdc"} = 45;
 
     # update lib/socket_wrapper/socket_wrapper.c
     #  #define MAX_WRAPPED_INTERFACES 64
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 0a04aa2..bbdbb9c 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -2180,6 +2180,7 @@ sub check_env($$)
 	ad_dc_no_ntlm        => [],
 	ad_dc_ntvfs          => [],
 	backupfromdc         => [],
+	customdc             => [],
 
 	fl2008r2dc           => ["ad_dc"],
 	fl2003dc             => ["ad_dc"],
@@ -3003,6 +3004,74 @@ sub setup_labdc
 	return $env;
 }
 
+# This spins up a custom testenv that can be based on any backup-file you want.
+# This is just intended for manual testing (rather than automated test-cases)
+sub setup_customdc
+{
+	my ($self, $prefix) = @_;
+	print "Preparing CUSTOM RESTORE DC...\n";
+	my $dc_name = "customdc";
+	my $password = "locDCpass1";
+	my $backup_file = $ENV{'BACKUP_FILE'};
+
+	# user must specify a backup file to restore via an ENV variable, i.e.
+	# BACKUP_FILE=backup-blah.tar.bz2 SELFTEST_TESTENV=customdc make testenv
+	if (not defined($backup_file)) {
+		warn("Please specify BACKUP_FILE");
+		return undef;
+	}
+
+	# allow user to specify a different domain/realm (but default to ad_dc)
+	my $domain = $ENV{'BACKUP_DOMAIN'};
+	if (not defined($domain)) {
+		$domain = "ADDOMAIN";
+	}
+	my $realm = $ENV{'BACKUP_REALM'};
+	if (not defined($realm)) {
+		$realm = "ADDOM.SAMBA.EXAMPLE.COM";
+	}
+
+	# create a placeholder directory and smb.conf, as well as the env vars
+	my ($env, $ctx) = $self->prepare_dc_testenv($prefix, $dc_name,
+						    $domain, $realm, $password);
+
+	# restore the specified backup file to populate the testenv
+	my $restore_dir = abs_path($prefix);
+	my $ret = $self->restore_backup_file($backup_file,
+					     "--newservername=$env->{SERVER}",
+					     $restore_dir, $env->{SERVERCONFFILE});
+	unless ($ret == 0) {
+		return undef;
+	}
+
+	# Change the admin password to the testenv default, just in case it's
+	# different, or in case this was a --no-secrets backup
+	my $samba_tool = Samba::bindir_path($self, "samba-tool");
+	my $cmd = "$samba_tool user setpassword $env->{USERNAME} ";
+	$cmd .= "--newpassword=$password -H $restore_dir/private/sam.ldb";
+
+	unless(system($cmd) == 0) {
+		warn("Failed to reset admin's password: \n$cmd");
+		return undef;
+	}
+
+	# re-create the testenv's krb5.conf (the restore may have overwritten it,
+	# if the backup-file was an offline backup)
+	Samba::mk_krb5_conf($ctx);
+
+	# start samba for the restored DC
+	if (not defined($self->check_or_start($env, "standard"))) {
+	    return undef;
+	}
+
+	my $upn_array = ["$env->{REALM}.upn"];
+	my $spn_array = ["$env->{REALM}.spn"];
+
+	$self->setup_namespaces($env, $upn_array, $spn_array);
+
+	return $env;
+}
+
 sub setup_none
 {
 	my ($self, $path) = @_;
-- 
2.7.4


From 2a9c919c5f0d62cd1f2206aecb9e6d53edf44bb9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue, 30 Oct 2018 13:06:20 +1300
Subject: [PATCH 2/3] dnsupdate: Pass smb.conf through to samba-tool commands

If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).

This patch passes through the smb.conf file, if one was specified.

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
 source4/scripting/bin/samba_dnsupdate | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index ae355e9..18b6197 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -122,6 +122,7 @@ for i in IPs:
     else:
         IP4s.append(i)
 
+smb_conf = sambaopts.get_loadparm_path()
 
 if opts.verbose:
     print("IPs: %s" % IPs)
@@ -620,6 +621,9 @@ def call_samba_tool(d, op="add", zone=None):
     if d.type == "NS":
         args = [rpc_server_ip, zone, short_name, "NS", d.dest]
 
+    if smb_conf and args:
+        args += ["--configfile=" + smb_conf]
+
     global error_count
     try:
         cmd = cmd_dns()
-- 
2.7.4


From 242aeb35ad7515d4771a7c7eca0c1cc0c8e3fdb9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue, 30 Oct 2018 14:11:46 +1300
Subject: [PATCH 3/3] dnsupdate: Skip kerberos step if use-file specified

If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.

This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).

Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Pair-Programmed-With: Garming Sam <garming at catalyst.net.nz>
---
 source4/scripting/bin/samba_dnsupdate | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index 18b6197..37c8c68 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -196,11 +196,11 @@ def get_credentials(lp):
     creds.set_krb_forwardable(credentials.NO_KRB_FORWARDABLE)
     (tmp_fd, ccachename) = tempfile.mkstemp()
     try:
-        creds.get_named_ccache(lp, ccachename)
-
         if opts.use_file is not None:
             return
 
+        creds.get_named_ccache(lp, ccachename)
+
         # Now confirm we can get a ticket to the DNS server
         get_krb5_rw_dns_server(creds, sub_vars['DNSDOMAIN'] + '.')
         return creds
-- 
2.7.4



More information about the samba-technical mailing list