[SCM] Samba Shared Repository - branch master updated
Andrew Bartlett
abartlet at samba.org
Thu Nov 1 12:07:03 UTC 2018
The branch, master has been updated
via e2ee595 selftest: Remove unnecessary code for backup testenvs
via 018ff49 dnsupdate: Skip kerberos step if use-file specified
via c35fb3a dnsupdate: Pass smb.conf through to samba-tool commands
via 1451b69 selftest: Add new customdc testenv that can load any backup-file
via d29d2f2 pytests: allow blackbox subcommands without a shell
from 790acef samba-tool ntacl: remove unused imports and variables
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit e2ee595202fb338e3ecc9bd8c37e1b89d33d091f
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu Nov 1 16:43:58 2018 +1300
selftest: Remove unnecessary code for backup testenvs
setup_namespaces() already gets done for the backupfromdc's domain, so
this step is unnecessary for the restoredc and offlinebackupdc testenvs
(which are based off the backupfromdc's database).
The setup_namespaces() step is still necessary for the renamedc/labdc,
as these don't have the UPN/SPN suffixes for the new realm yet.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Nov 1 13:06:01 CET 2018 on sn-devel-144
commit 018ff496e4be6ed536e56b2e6b1940c048f09762
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Oct 30 14:11:46 2018 +1300
dnsupdate: Skip kerberos step if use-file specified
If there's a problem in get_credentials() (getting the machine account
Kerberos credentials), then we fallback to use_samba_tool (essentially
ignoring use-file). However, there's no need to do this, as use-file
shouldn't require Kerberos credentials.
This was making bootstrapping issues starting a testenv harder to debug.
Obviously, Kerberos is dependent on DNS functioning correctly, but
running dnsupdate was also dependent on having a working Kerberos KDC.
In my case, the testenv had a bad krb5.conf file, but the problem
appeared as resolv-wrapper errors (due to a missing RESOLV_WRAPPER_HOSTS
file, which should've been generated by dnsupdate).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Pair-Programmed-With: Garming Sam <garming at catalyst.net.nz>
commit c35fb3a88b5c92c343d09f1a707806ea6e46db60
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Oct 30 13:06:20 2018 +1300
dnsupdate: Pass smb.conf through to samba-tool commands
If you call samba_dnsupdate with a --configfile option, this wasn't
passed through to the samba-tool commands the script tries to run.
Normally, samba_dnsupdate would only be run on the DC itself, so it
shouldn't be a big deal, however, this may be a problem if you install
the samba database into a non-default location (i.e. not
/usr/local/samba).
This patch passes through the smb.conf file, if one was specified.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 1451b696ee332b49edf691221b97962c95a10fd6
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri Oct 26 11:08:54 2018 +1300
selftest: Add new customdc testenv that can load any backup-file
This adds a new testenv that can be used for sandpit/manual testing.
This testenv can be based off any backup-file that you like.
The main use case is large databases. Populating 1000s of users is
time-consuming (it can take hours to create a really large DB). Instead
of having to manually add users to the testenv every time you want to
try something, this allows you to populate the users just once, take a
backup/snapshot of the DB, and then spin up the backup multiple times.
In theory this testenv could be useful for other situations too, e.g.
dealing with a corrupted database, testing DB migration (e.g. 4.7 -->
4.8), or if (for some reason) you wanted to create a realistic
lab-domain within a testenv.
To run-up the testenv you need to specify a BACKUP_FILE environment
variable (the same way we specify the SELFTEST_TESTENV), e.g.
BACKUP_FILE=/files/backup-10k-ad_dc.tar.bz2 \
SELFTEST_TESTENV=customdc make testenv
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit d29d2f2f70223e585eb645caa8bfc18993c47fdc
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Oct 4 16:46:34 2018 +1300
pytests: allow blackbox subcommands without a shell
When given a list, it will use the list directly as an argument list,
avoiding shell-expansion and the intermediatory process.
This removes shell expansion trouble, and saves the machine a little
bit of work.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
python/samba/tests/__init__.py | 46 ++++++++++----
selftest/target/Samba.pm | 1 +
selftest/target/Samba4.pm | 110 ++++++++++++++++++++++++++++++----
source4/scripting/bin/samba_dnsupdate | 8 ++-
4 files changed, 140 insertions(+), 25 deletions(-)
Changeset truncated at 500 lines:
diff --git a/python/samba/tests/__init__.py b/python/samba/tests/__init__.py
index 7957a1a..66067fc 100644
--- a/python/samba/tests/__init__.py
+++ b/python/samba/tests/__init__.py
@@ -56,6 +56,9 @@ except ImportError:
class SkipTest(Exception):
"""Test skipped."""
+BINDIR = os.path.abspath(os.path.join(os.path.dirname(__file__),
+ "../../../../bin"))
+
HEXDUMP_FILTER = bytearray([x if ((len(repr(chr(x))) == 3) and (x < 127)) else ord('.') for x in range(256)])
@@ -348,14 +351,20 @@ class BlackboxProcessError(Exception):
def __init__(self, returncode, cmd, stdout, stderr, msg=None):
self.returncode = returncode
- self.cmd = cmd
+ if isinstance(cmd, list):
+ self.cmd = ' '.join(cmd)
+ self.shell = False
+ else:
+ self.cmd = cmd
+ self.shell = True
self.stdout = stdout
self.stderr = stderr
self.msg = msg
def __str__(self):
- s = ("Command '%s'; exit status %d; stdout: '%s'; stderr: '%s'" %
- (self.cmd, self.returncode, self.stdout, self.stderr))
+ s = ("Command '%s'; shell %s; exit status %d; "
+ "stdout: '%s'; stderr: '%s'" %
+ (self.cmd, self.shell, self.returncode, self.stdout, self.stderr))
if self.msg is not None:
s = "%s; message: %s" % (s, self.msg)
@@ -366,14 +375,22 @@ class BlackboxTestCase(TestCaseInTempDir):
"""Base test case for blackbox tests."""
def _make_cmdline(self, line):
- bindir = os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../../bin"))
- parts = line.split(" ")
- if os.path.exists(os.path.join(bindir, parts[0])):
- cmd = parts[0]
- parts[0] = os.path.join(bindir, parts[0])
- if cmd == "samba-tool" and os.getenv("PYTHON", None):
- parts = [os.environ["PYTHON"]] + parts
- line = " ".join(parts)
+ """Expand the called script into a fully resolved path in the bin
+ directory."""
+ if isinstance(line, list):
+ parts = line
+ else:
+ parts = line.split(" ", 1)
+ cmd = parts[0]
+ exe = os.path.join(BINDIR, cmd)
+ if os.path.exists(exe):
+ parts[0] = exe
+ if cmd =='samba-tool' and os.getenv("PYTHON", None):
+ parts.insert(0, os.environ["PYTHON"])
+
+ if not isinstance(line, list):
+ line = " ".join(parts)
+
return line
def check_run(self, line, msg=None):
@@ -381,10 +398,11 @@ class BlackboxTestCase(TestCaseInTempDir):
def check_exit_code(self, line, expected, msg=None):
line = self._make_cmdline(line)
+ use_shell = not isinstance(line, list)
p = subprocess.Popen(line,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
- shell=True)
+ shell=use_shell)
stdoutdata, stderrdata = p.communicate()
retcode = p.returncode
if retcode != expected:
@@ -395,8 +413,10 @@ class BlackboxTestCase(TestCaseInTempDir):
msg)
def check_output(self, line):
+ use_shell = not isinstance(line, list)
line = self._make_cmdline(line)
- p = subprocess.Popen(line, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, close_fds=True)
+ p = subprocess.Popen(line, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+ shell=use_shell, close_fds=True)
stdoutdata, stderrdata = p.communicate()
retcode = p.returncode
if retcode:
diff --git a/selftest/target/Samba.pm b/selftest/target/Samba.pm
index a3be713..aa6ec9e 100644
--- a/selftest/target/Samba.pm
+++ b/selftest/target/Samba.pm
@@ -423,6 +423,7 @@ sub get_interface($)
$interfaces{"renamedc"} = 42;
$interfaces{"labdc"} = 43;
$interfaces{"offlinebackupdc"} = 44;
+ $interfaces{"customdc"} = 45;
# update lib/socket_wrapper/socket_wrapper.c
# #define MAX_WRAPPED_INTERFACES 64
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 770ba4f..41e550b 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -12,6 +12,7 @@ use POSIX;
use SocketWrapper;
use target::Samba;
use target::Samba3;
+use Archive::Tar;
sub new($$$$$) {
my ($classname, $bindir, $ldap, $srcdir, $server_maxtime) = @_;
@@ -2180,6 +2181,7 @@ sub check_env($$)
ad_dc_no_ntlm => [],
ad_dc_ntvfs => [],
backupfromdc => [],
+ customdc => [],
fl2008r2dc => ["ad_dc"],
fl2003dc => ["ad_dc"],
@@ -2841,11 +2843,6 @@ sub setup_restoredc
return undef;
}
- my $upn_array = ["$env->{REALM}.upn"];
- my $spn_array = ["$env->{REALM}.spn"];
-
- $self->setup_namespaces($env, $upn_array, $spn_array);
-
return $env;
}
@@ -2937,11 +2934,6 @@ sub setup_offlinebackupdc
return undef;
}
- my $upn_array = ["$env->{REALM}.upn"];
- my $spn_array = ["$env->{REALM}.spn"];
-
- $self->setup_namespaces($env, $upn_array, $spn_array);
-
return $env;
}
@@ -3004,6 +2996,104 @@ sub setup_labdc
return $env;
}
+# Inspects a backup *.tar.bz2 file and determines the realm/domain it contains
+sub get_backup_domain_realm
+{
+ my ($self, $backup_file) = @_;
+
+ print "Determining REALM/DOMAIN values in backup...\n";
+
+ # The backup will have the correct domain/realm values in the smb.conf.
+ # So we can work out the env variables the testenv should use based on
+ # that. Let's start by extracting the smb.conf
+ my $tar = Archive::Tar->new($backup_file);
+ my $tmpdir = File::Temp->newdir();
+ my $smbconf = "$tmpdir/smb.conf";
+
+ # note that the filepaths within the tar-file differ slightly for online
+ # and offline backups
+ if ($tar->contains_file("etc/smb.conf")) {
+ $tar->extract_file("etc/smb.conf", $smbconf);
+ } elsif ($tar->contains_file("./etc/smb.conf")) {
+ $tar->extract_file("./etc/smb.conf", $smbconf);
+ } else {
+ warn("Could not find smb.conf in $backup_file");
+ return undef, undef;
+ }
+
+ # now use testparm to read the values we're interested in
+ my $testparm = Samba::bindir_path($self, "testparm");
+ my $domain = `$testparm $smbconf -sl --parameter-name=WORKGROUP`;
+ my $realm = `$testparm $smbconf -sl --parameter-name=REALM`;
+ chomp $realm;
+ chomp $domain;
+ print "Backup-file REALM is $realm, DOMAIN is $domain\n";
+
+ return ($domain, $realm);
+}
+
+# This spins up a custom testenv that can be based on any backup-file you want.
+# This is just intended for manual testing (rather than automated test-cases)
+sub setup_customdc
+{
+ my ($self, $prefix) = @_;
+ print "Preparing CUSTOM RESTORE DC...\n";
+ my $dc_name = "customdc";
+ my $password = "locDCpass1";
+ my $backup_file = $ENV{'BACKUP_FILE'};
+
+ # user must specify a backup file to restore via an ENV variable, i.e.
+ # BACKUP_FILE=backup-blah.tar.bz2 SELFTEST_TESTENV=customdc make testenv
+ if (not defined($backup_file)) {
+ warn("Please specify BACKUP_FILE");
+ return undef;
+ }
+
+ # work out the correct domain/realm env values from the backup-file
+ my ($domain, $realm) = $self->get_backup_domain_realm($backup_file);
+
+ # create a placeholder directory and smb.conf, as well as the env vars.
+ my ($env, $ctx) = $self->prepare_dc_testenv($prefix, $dc_name,
+ $domain, $realm, $password);
+
+ # restore the specified backup file to populate the testenv
+ my $restore_dir = abs_path($prefix);
+ my $ret = $self->restore_backup_file($backup_file,
+ "--newservername=$env->{SERVER}",
+ $restore_dir, $env->{SERVERCONFFILE});
+ unless ($ret == 0) {
+ return undef;
+ }
+
+ # Change the admin password to the testenv default, just in case it's
+ # different, or in case this was a --no-secrets backup
+ my $samba_tool = Samba::bindir_path($self, "samba-tool");
+ my $cmd = "$samba_tool user setpassword $env->{USERNAME} ";
+ $cmd .= "--newpassword=$password -H $restore_dir/private/sam.ldb";
+
+ unless(system($cmd) == 0) {
+ warn("Failed to reset admin's password: \n$cmd");
+ return undef;
+ }
+
+ # re-create the testenv's krb5.conf (the restore may have overwritten it,
+ # if the backup-file was an offline backup)
+ Samba::mk_krb5_conf($ctx);
+
+ # start samba for the restored DC
+ if (not defined($self->check_or_start($env, "standard"))) {
+ return undef;
+ }
+
+ # if this was a backup-rename, then we may need to setup namespaces
+ my $upn_array = ["$env->{REALM}.upn"];
+ my $spn_array = ["$env->{REALM}.spn"];
+
+ $self->setup_namespaces($env, $upn_array, $spn_array);
+
+ return $env;
+}
+
sub setup_none
{
my ($self, $path) = @_;
diff --git a/source4/scripting/bin/samba_dnsupdate b/source4/scripting/bin/samba_dnsupdate
index ae355e9..37c8c68 100755
--- a/source4/scripting/bin/samba_dnsupdate
+++ b/source4/scripting/bin/samba_dnsupdate
@@ -122,6 +122,7 @@ for i in IPs:
else:
IP4s.append(i)
+smb_conf = sambaopts.get_loadparm_path()
if opts.verbose:
print("IPs: %s" % IPs)
@@ -195,11 +196,11 @@ def get_credentials(lp):
creds.set_krb_forwardable(credentials.NO_KRB_FORWARDABLE)
(tmp_fd, ccachename) = tempfile.mkstemp()
try:
- creds.get_named_ccache(lp, ccachename)
-
if opts.use_file is not None:
return
+ creds.get_named_ccache(lp, ccachename)
+
# Now confirm we can get a ticket to the DNS server
get_krb5_rw_dns_server(creds, sub_vars['DNSDOMAIN'] + '.')
return creds
@@ -620,6 +621,9 @@ def call_samba_tool(d, op="add", zone=None):
if d.type == "NS":
args = [rpc_server_ip, zone, short_name, "NS", d.dest]
+ if smb_conf and args:
+ args += ["--configfile=" + smb_conf]
+
global error_count
try:
cmd = cmd_dns()
--
Samba Shared Repository
More information about the samba-cvs
mailing list