[PATCH] Create a custom testenv from backup-file for manual testing
Tim Beale
timbeale at catalyst.net.nz
Tue Oct 30 20:52:19 UTC 2018
The attached patch allows you to easily spin up any custom testenv that
you want, for manual testing. All you need is a backup-file that the
testenv DC will be based off.
The main use-case is any testing that involves a large database.
Creating user accounts is slow. Even with the improvements to the
traffic_replay user generation, creating 5,000 users still takes close
to 10 minutes. Instead of creating a blank testenv and slowly populating
it with users, you can do this step once, take a backup, and then spin
up the backup database multiple times as a new testenv.
This testenv might be useful in other situations too. E.g. testing
migrating databases across samba versions, you spot a DB corruption and
want to try diagnosing/fixing it without completely losing the original
problem state, you want to create a labdc-replica testenv of a real
network, etc.
Also fixed a couple of minor dnsupdate problems that were making
debugging testenv bootstrapping issues difficult.
CI pass: https://gitlab.com/catalyst-samba/samba/pipelines/34756736
-------------- next part --------------
From e4f6ee4680637e14d224d3774193ef1f4b4ecdd9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 26 Oct 2018 11:08:54 +1300
Subject: [PATCH 1/3] selftest: Add new customdc testenv that can load any
backup-file
This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.
The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.
In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.
To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
SELFTEST_TESTENV=customdc make testenv
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
selftest/target/Samba.pm | 1 +
selftest/target/Samba4.pm | 69 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 70 insertions(+)
diff --git a/selftest/target/Samba.pm b/selftest/target/Samba.pm
index a3be713..aa6ec9e 100644
--- a/selftest/target/Samba.pm
+++ b/selftest/target/Samba.pm
@@ -423,6 +423,7 @@ sub get_interface($)
$interfaces{"renamedc"} = 42;
$interfaces{"labdc"} = 43;
$interfaces{"offlinebackupdc"} = 44;
+ $interfaces{"customdc"} = 45;
# update lib/socket_wrapper/socket_wrapper.c
# #define MAX_WRAPPED_INTERFACES 64
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 0a04aa2..bbdbb9c 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -2180,6 +2180,7 @@ sub check_env($$)
ad_dc_no_ntlm => [],
ad_dc_ntvfs => [],
backupfromdc => [],
+ customdc => [],
fl2008r2dc => ["ad_dc"],
fl2003dc => ["ad_dc"],
@@ -3003,6 +3004,74 @@ sub setup_labdc
return $env;
}
+# This spins up a custom testenv that can be based on any backup-file you want.
+# This is just intended for manual testing (rather than automated test-cases)
+sub setup_customdc
+{
+ my ($self, $prefix) = @_;
+ print "Preparing CUSTOM RESTORE DC...\n";
+ my $dc_name = "customdc";
+ my $password = "locDCpass1";
+ my $backup_file = $ENV{'BACKUP_FILE'};
+
+ # user must specify a backup file to restore via an ENV variable, i.e.
+ # BACKUP_FILE=backup-blah.tar.bz2 SELFTEST_TESTENV=customdc make testenv
+ if (not defined($backup_file)) {
+ warn("Please specify BACKUP_FILE");
+ return undef;
+ }
+
+ # allow user to specify a different domain/realm (but default to ad_dc)
+ my $domain = $ENV{'BACKUP_DOMAIN'};
+ if (not defined($domain)) {
+ $domain = "ADDOMAIN";
+ }
+ my $realm = $ENV{'BACKUP_REALM'};
+ if (not defined($realm)) {
+ $realm = "ADDOM.SAMBA.EXAMPLE.COM";
+ }
+
+ # create a placeholder directory and smb.conf, as well as the env vars
+ my ($env, $ctx) = $self->prepare_dc_testenv($prefix, $dc_name,
+ $domain, $realm, $password);
+
+ # restore the specified backup file to populate the testenv
+ my $restore_dir = abs_path($prefix);
+ my $ret = $self->restore_backup_file($backup_file,
+ "--newservername=$env->{SERVER}",
+ $restore_dir, $env->{SERVERCONFFILE});
+ unless ($ret == 0) {
+ return undef;
+ }
+
+ # Change the admin password to the testenv default, just in case it's
+ # different, or in case this was a --no-secrets backup
+ my $samba_tool = Samba::bindir_path($self, "samba-tool");
+ my $cmd = "$samba_tool user setpassword $env->{USERNAME} ";
+ $cmd .= "--newpassword=$password -H $restore_dir/private/sam.ldb";
+
+ unless(system($cmd) == 0) {
+ warn("Failed to reset admin's password: \n$cmd");
+ return undef;
+ }
+
+ # re-create the testenv's krb5.conf (the restore may have overwritten it,
+ # if the backup-file was an offline backup)
+ Samba::mk_krb5_conf($ctx);
+
+ # start samba for the restored DC
+ if (not defined($self->check_or_start($env, "standard"))) {
+ return undef;
+ }
+
+ my $upn_array = ["$env->{REALM}.upn"];
+ my $spn_array = ["$env->{REALM}.spn"];
+
+ $self->setup_namespaces($env, $upn_array, $spn_array);
+
+ return $env;
+}
+
sub setup_none
{
my ($self, $path) = @_;
--
2.7.4
From 2a9c919c5f0d62cd1f2206aecb9e6d53edf44bb9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue, 30 Oct 2018 13:06:20 +1300
Subject: [PATCH 2/3] dnsupdate: Pass smb.conf through to samba-tool commands
If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).
This patch passes through the smb.conf file, if one was specified.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
source4/scripting/bin/samba_dnsupdate | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index ae355e9..18b6197 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -122,6 +122,7 @@ for i in IPs:
else:
IP4s.append(i)
+smb_conf = sambaopts.get_loadparm_path()
if opts.verbose:
print("IPs: %s" % IPs)
@@ -620,6 +621,9 @@ def call_samba_tool(d, op="add", zone=None):
if d.type == "NS":
args = [rpc_server_ip, zone, short_name, "NS", d.dest]
+ if smb_conf and args:
+ args += ["--configfile=" + smb_conf]
+
global error_count
try:
cmd = cmd_dns()
--
2.7.4
From 242aeb35ad7515d4771a7c7eca0c1cc0c8e3fdb9 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue, 30 Oct 2018 14:11:46 +1300
Subject: [PATCH 3/3] dnsupdate: Skip kerberos step if use-file specified
If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.
This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Pair-Programmed-With: Garming Sam <garming at catalyst.net.nz>
---
source4/scripting/bin/samba_dnsupdate | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index 18b6197..37c8c68 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -196,11 +196,11 @@ def get_credentials(lp):
creds.set_krb_forwardable(credentials.NO_KRB_FORWARDABLE)
(tmp_fd, ccachename) = tempfile.mkstemp()
try:
- creds.get_named_ccache(lp, ccachename)
-
if opts.use_file is not None:
return
+ creds.get_named_ccache(lp, ccachename)
+
# Now confirm we can get a ticket to the DNS server
get_krb5_rw_dns_server(creds, sub_vars['DNSDOMAIN'] + '.')
return creds
--
2.7.4
More information about the samba-technical
mailing list