[SCM] Samba Shared Repository - branch master updated
Andrew Tridgell
tridge at samba.org
Sat Sep 25 19:21:53 MDT 2010
The branch, master has been updated
via 69ef2b3 upgradeprovision: use the same case for hostname in reference provision as in the current provision
via e956072 autobuild: added --retry option
via e8fec1d s4-dns: the DNS/${HOSTNAME} SPN should be on the DNS account only
via b8444b6 s4-provision: switch to dns-HOSTNAME instead of dns
via 0079165 autobuild: enable ccache
via d9a0546 autobuild: added --fix-whitespace option
via 5d5ae09 autobuild: added --rebase-master and --push-master
via 91b62a2 autobuild: added a EDITOR script to mark successful autobuilds
via f7fb272 autobuild: added --rebase option
via a09b241 script: added autobuild.py automatic build test script
from f1b3c4d s4-possibleinferiors.py: Fix usage of 'paged_search' module for remote LDB connections
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 69ef2b37050d44394c4e68c315a5af0afe332656
Author: Matthieu Patou <mat at matws.net>
Date: Sat Sep 25 21:13:07 2010 +0400
upgradeprovision: use the same case for hostname in reference provision as in the current provision
Autobuild-User: Andrew Tridgell <tridge at samba.org>
Autobuild-Date: Sun Sep 26 01:21:52 UTC 2010 on sn-devel-104
commit e9560722ca01b3a2f23f5750292c0e6450141675
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 17:35:55 2010 -0700
autobuild: added --retry option
this allows the build to auto retry if master changes
commit e8fec1d3c6c9b8b08de45ad724fe1eedb22fba7d
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 00:07:57 2010 -0700
s4-dns: the DNS/${HOSTNAME} SPN should be on the DNS account only
commit b8444b64a32d698b01acce2a1307723cc69a472b
Author: Andrew Tridgell <tridge at samba.org>
Date: Fri Sep 24 23:25:49 2010 -0700
s4-provision: switch to dns-HOSTNAME instead of dns
We now use a host specific account name for the DNS account, which is
the account used for dynamic DNS updates. We also setup the
servicePrincipalName for automatic update, and add both DNS/${DNSDOMAIN}
and DNS/${DNSNAME} for compatibility with both the old and new SPNs
Pair-Programmed-With: Andrew Bartlett <abartlet at samba.org>
commit 00791652f4a4894ecffbca38d1f9bb4584fb2635
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 16:32:55 2010 -0700
autobuild: enable ccache
commit d9a0546666c4628dbf577c6525ad64f3ee3ef077
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 16:18:00 2010 -0700
autobuild: added --fix-whitespace option
commit 5d5ae099d16d1902e4a85adec5b1e6a9085b6118
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 15:53:32 2010 -0700
autobuild: added --rebase-master and --push-master
Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
commit 91b62a274411477f67f50f0f653dd17bf1e65c1d
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 15:30:13 2010 -0700
autobuild: added a EDITOR script to mark successful autobuilds
Pair-Programmed-With: Stefan Metzmacher <metze at samba.org>
commit f7fb272e953b743ca9830618af9b1c290701ffec
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 14:09:11 2010 -0700
autobuild: added --rebase option
this allows you to rebase on a URL before starting
commit a09b241ebb04dbc0285e5bee0e67027e055f719a
Author: Andrew Tridgell <tridge at samba.org>
Date: Sat Sep 25 12:51:51 2010 -0700
script: added autobuild.py automatic build test script
this can be used to run all tests in parallel, in free git clones
-----------------------------------------------------------------------
Summary of changes:
script/autobuild.py | 363 ++++++++++++++++++++++
script/commit_mark.sh | 14 +
source4/scripting/python/samba/provision.py | 30 ++-
source4/scripting/python/samba/upgradehelpers.py | 2 +-
source4/setup/provision_dns_add.ldif | 10 +-
source4/setup/secrets_dns.ldif | 5 +-
source4/setup/secrets_self_join.ldif | 13 -
source4/setup/spn_update_list | 1 -
8 files changed, 411 insertions(+), 27 deletions(-)
create mode 100755 script/autobuild.py
create mode 100755 script/commit_mark.sh
delete mode 100644 source4/setup/secrets_self_join.ldif
Changeset truncated at 500 lines:
diff --git a/script/autobuild.py b/script/autobuild.py
new file mode 100755
index 0000000..4941562
--- /dev/null
+++ b/script/autobuild.py
@@ -0,0 +1,363 @@
+#!/usr/bin/env python
+# run tests on all Samba subprojects and push to a git tree on success
+# Copyright Andrew Tridgell 2010
+# released under GNU GPL v3 or later
+
+from subprocess import Popen, PIPE
+import os, signal, tarfile, sys, time
+from optparse import OptionParser
+
+
+samba_master = os.getenv('SAMBA_MASTER', 'git://git.samba.org/samba.git')
+samba_master_ssh = os.getenv('SAMBA_MASTER_SSH', 'git+ssh://git.samba.org/data/git/samba.git')
+
+cleanup_list = []
+
+os.putenv('CC', "ccache gcc")
+
+tasks = {
+ "source3" : [ "./autogen.sh",
+ "./configure.developer ${PREFIX}",
+ "make basics",
+ "make -j 4 everything", # don't use too many processes
+ "make install",
+ "TDB_NO_FSYNC=1 make test" ],
+
+ "source4" : [ "./autogen.sh",
+ "./configure.developer ${PREFIX}",
+ "make -j",
+ "make install",
+ "TDB_NO_FSYNC=1 make test" ],
+
+ "source4/lib/ldb" : [ "./autogen-waf.sh",
+ "./configure --enable-developer -C ${PREFIX}",
+ "make -j",
+ "make install",
+ "make test" ],
+
+ "lib/tdb" : [ "./autogen-waf.sh",
+ "./configure --enable-developer -C ${PREFIX}",
+ "make -j",
+ "make install",
+ "make test" ],
+
+ "lib/talloc" : [ "./autogen-waf.sh",
+ "./configure --enable-developer -C ${PREFIX}",
+ "make -j",
+ "make install",
+ "make test" ],
+
+ "lib/replace" : [ "./autogen-waf.sh",
+ "./configure --enable-developer -C ${PREFIX}",
+ "make -j",
+ "make install",
+ "make test" ],
+
+ "lib/tevent" : [ "./autogen-waf.sh",
+ "./configure --enable-developer -C ${PREFIX}",
+ "make -j",
+ "make install",
+ "make test" ],
+}
+
+retry_task = [ '''set -e
+ git remote add -t master master %s
+ while :; do
+ sleep 60
+ git fetch master
+ git describe > HEAD.desc
+ git describe > master.desc
+ diff HEAD.desc master.desc
+ done
+ ''' % samba_master]
+
+def run_cmd(cmd, dir=".", show=None):
+ cwd = os.getcwd()
+ os.chdir(dir)
+ if show is None:
+ show = options.verbose
+ if show:
+ print("Running: '%s' in '%s'" % (cmd, dir))
+ ret = os.system(cmd)
+ os.chdir(cwd)
+ if ret != 0:
+ raise Exception("FAILED %s: %d" % (cmd, ret))
+
+class builder:
+ '''handle build of one directory'''
+ def __init__(self, name, sequence):
+ self.name = name
+
+ if name in ['pass', 'fail', 'retry']:
+ self.dir = "."
+ else:
+ self.dir = self.name
+
+ self.tag = self.name.replace('/', '_')
+ self.sequence = sequence
+ self.next = 0
+ self.stdout_path = "%s/%s.stdout" % (testbase, self.tag)
+ self.stderr_path = "%s/%s.stderr" % (testbase, self.tag)
+ cleanup_list.append(self.stdout_path)
+ cleanup_list.append(self.stderr_path)
+ run_cmd("rm -f %s %s" % (self.stdout_path, self.stderr_path))
+ self.stdout = open(self.stdout_path, 'w')
+ self.stderr = open(self.stderr_path, 'w')
+ self.stdin = open("/dev/null", 'r')
+ self.sdir = "%s/%s" % (testbase, self.tag)
+ self.prefix = "%s/prefix/%s" % (testbase, self.tag)
+ run_cmd("rm -rf %s" % self.sdir)
+ cleanup_list.append(self.sdir)
+ cleanup_list.append(self.prefix)
+ os.makedirs(self.sdir)
+ run_cmd("rm -rf %s" % self.sdir)
+ run_cmd("git clone --shared %s %s" % (gitroot, self.sdir))
+ self.start_next()
+
+ def start_next(self):
+ if self.next == len(self.sequence):
+ print '%s: Completed OK' % self.name
+ self.done = True
+ return
+ self.cmd = self.sequence[self.next].replace("${PREFIX}", "--prefix=%s" % self.prefix)
+ print '%s: Running %s' % (self.name, self.cmd)
+ cwd = os.getcwd()
+ os.chdir("%s/%s" % (self.sdir, self.dir))
+ self.proc = Popen(self.cmd, shell=True,
+ stdout=self.stdout, stderr=self.stderr, stdin=self.stdin)
+ os.chdir(cwd)
+ self.next += 1
+
+
+class buildlist:
+ '''handle build of multiple directories'''
+ def __init__(self, tasklist, tasknames):
+ global tasks
+ self.tlist = []
+ self.tail_proc = None
+ self.retry = None
+ if tasknames == ['pass']:
+ tasks = { 'pass' : [ '/bin/true' ]}
+ if tasknames == ['fail']:
+ tasks = { 'fail' : [ '/bin/false' ]}
+ if tasknames == []:
+ tasknames = tasklist
+ for n in tasknames:
+ b = builder(n, tasks[n])
+ self.tlist.append(b)
+ if options.retry:
+ self.retry = builder('retry', retry_task)
+ self.need_retry = False
+
+ def kill_kids(self):
+ if self.tail_proc is not None:
+ self.tail_proc.terminate()
+ self.tail_proc.wait()
+ self.tail_proc = None
+ if self.retry is not None:
+ self.retry.proc.terminate()
+ self.retry.proc.wait()
+ self.retry = None
+ for b in self.tlist:
+ if b.proc is not None:
+ b.proc.terminate()
+ b.proc.wait()
+ b.proc = None
+
+ def wait_one(self):
+ while True:
+ none_running = True
+ for b in self.tlist:
+ if b.proc is None:
+ continue
+ none_running = False
+ b.status = b.proc.poll()
+ if b.status is None:
+ continue
+ b.proc = None
+ return b
+ if options.retry:
+ ret = self.retry.proc.poll()
+ if ret is not None:
+ self.need_retry = True
+ self.retry = None
+ return None
+ if none_running:
+ return None
+ time.sleep(0.1)
+
+ def run(self):
+ while True:
+ b = self.wait_one()
+ if options.retry and self.need_retry:
+ self.kill_kids()
+ print("retry needed")
+ return (0, "retry")
+ if b is None:
+ break
+ if os.WIFSIGNALED(b.status) or os.WEXITSTATUS(b.status) != 0:
+ self.kill_kids()
+ return (b.status, "%s: failed '%s' with status %d" % (b.name, b.cmd, b.status))
+ b.start_next()
+ self.kill_kids()
+ return (0, "All OK")
+
+ def tarlogs(self, fname):
+ tar = tarfile.open(fname, "w:gz")
+ for b in self.tlist:
+ tar.add(b.stdout_path, arcname="%s.stdout" % b.tag)
+ tar.add(b.stderr_path, arcname="%s.stderr" % b.tag)
+ tar.close()
+
+ def remove_logs(self):
+ for b in self.tlist:
+ os.unlink(b.stdout_path)
+ os.unlink(b.stderr_path)
+
+ def start_tail(self):
+ cwd = os.getcwd()
+ cmd = "tail -f *.stdout *.stderr"
+ os.chdir(testbase)
+ self.tail_proc = Popen(cmd, shell=True)
+ os.chdir(cwd)
+
+
+def cleanup():
+ if options.nocleanup:
+ return
+ print("Cleaning up ....")
+ for d in cleanup_list:
+ run_cmd("rm -rf %s" % d)
+
+
+def find_git_root():
+ '''get to the top of the git repo'''
+ cwd=os.getcwd()
+ while os.getcwd() != '/':
+ try:
+ os.stat(".git")
+ ret = os.getcwd()
+ os.chdir(cwd)
+ return ret
+ except:
+ os.chdir("..")
+ pass
+ os.chdir(cwd)
+ return None
+
+def rebase_tree(url):
+ print("Rebasing on %s" % url)
+ run_cmd("git remote add -t master master %s" % url, show=True, dir=test_master)
+ run_cmd("git fetch master", show=True, dir=test_master)
+ if options.fix_whitespace:
+ run_cmd("git rebase --whitespace=fix master/master", show=True, dir=test_master)
+ else:
+ run_cmd("git rebase master/master", show=True, dir=test_master)
+
+def push_to(url):
+ print("Pushing to %s" % url)
+ if options.mark:
+ run_cmd("EDITOR=script/commit_mark.sh git commit --amend -c HEAD", dir=test_master)
+ run_cmd("git remote add -t master pushto %s" % url, show=True, dir=test_master)
+ run_cmd("git push pushto +HEAD:master", show=True, dir=test_master)
+
+def_testbase = os.getenv("AUTOBUILD_TESTBASE", "/memdisk/%s" % os.getenv('USER'))
+
+parser = OptionParser()
+parser.add_option("", "--tail", help="show output while running", default=False, action="store_true")
+parser.add_option("", "--keeplogs", help="keep logs", default=False, action="store_true")
+parser.add_option("", "--nocleanup", help="don't remove test tree", default=False, action="store_true")
+parser.add_option("", "--testbase", help="base directory to run tests in (default %s)" % def_testbase,
+ default=def_testbase)
+parser.add_option("", "--passcmd", help="command to run on success", default=None)
+parser.add_option("", "--verbose", help="show all commands as they are run",
+ default=False, action="store_true")
+parser.add_option("", "--rebase", help="rebase on the given tree before testing",
+ default=None, type='str')
+parser.add_option("", "--rebase-master", help="rebase on %s before testing" % samba_master,
+ default=False, action='store_true')
+parser.add_option("", "--pushto", help="push to a git url on success",
+ default=None, type='str')
+parser.add_option("", "--push-master", help="push to %s on success" % samba_master_ssh,
+ default=False, action='store_true')
+parser.add_option("", "--mark", help="add a Tested-By signoff before pushing",
+ default=False, action="store_true")
+parser.add_option("", "--fix-whitespace", help="fix whitespace on rebase",
+ default=False, action="store_true")
+parser.add_option("", "--retry", help="automatically retry if master changes",
+ default=False, action="store_true")
+
+
+(options, args) = parser.parse_args()
+
+if options.retry:
+ if not options.rebase_master and options.rebase is None:
+ raise Exception('You can only use --retry if you also rebase')
+
+testbase = "%s/build.%u" % (options.testbase, os.getpid())
+test_master = "%s/master" % testbase
+
+gitroot = find_git_root()
+if gitroot is None:
+ raise Exception("Failed to find git root")
+
+try:
+ os.makedirs(testbase)
+except Exception, reason:
+ raise Exception("Unable to create %s : %s" % (testbase, reason))
+cleanup_list.append(testbase)
+
+while True:
+ try:
+ run_cmd("rm -rf %s" % test_master)
+ cleanup_list.append(test_master)
+ run_cmd("git clone --shared %s %s" % (gitroot, test_master))
+ except:
+ cleanup()
+ raise
+
+ try:
+ if options.rebase is not None:
+ rebase_tree(options.rebase)
+ elif options.rebase_master:
+ rebase_tree(samba_master)
+ blist = buildlist(tasks, args)
+ if options.tail:
+ blist.start_tail()
+ (status, errstr) = blist.run()
+ if status != 0 or errstr != "retry":
+ break
+ cleanup()
+ except:
+ cleanup()
+ raise
+
+blist.kill_kids()
+if options.tail:
+ print("waiting for tail to flush")
+ time.sleep(1)
+
+if status == 0:
+ print errstr
+ if options.passcmd is not None:
+ print("Running passcmd: %s" % options.passcmd)
+ run_cmd(options.passcmd, dir=test_master)
+ if options.pushto is not None:
+ push_to(options.pushto)
+ elif options.push_master:
+ push_to(samba_master_ssh)
+ if options.keeplogs:
+ blist.tarlogs("logs.tar.gz")
+ print("Logs in logs.tar.gz")
+ blist.remove_logs()
+ cleanup()
+ print(errstr)
+ sys.exit(0)
+
+# something failed, gather a tar of the logs
+blist.tarlogs("logs.tar.gz")
+blist.remove_logs()
+cleanup()
+print(errstr)
+print("Logs in logs.tar.gz")
+sys.exit(os.WEXITSTATUS(status))
diff --git a/script/commit_mark.sh b/script/commit_mark.sh
new file mode 100755
index 0000000..bb1e942
--- /dev/null
+++ b/script/commit_mark.sh
@@ -0,0 +1,14 @@
+#!/bin/sh
+# add a autobuild message to the HEAD commit
+
+if grep -q '^Autobuild.User' "$1"; then
+ echo "Already marked as tested"
+ exit 0
+fi
+
+fullname=$(getent passwd $USER | cut -d: -f5| cut -d',' -f1)
+cat <<EOF >> "$1"
+Autobuild-User: $fullname <$USER at samba.org>
+Autobuild-Date: $(date) on $(hostname)
+EOF
+exit 0
diff --git a/source4/scripting/python/samba/provision.py b/source4/scripting/python/samba/provision.py
index 7d8b08f..c1a35c9 100644
--- a/source4/scripting/python/samba/provision.py
+++ b/source4/scripting/python/samba/provision.py
@@ -687,21 +687,26 @@ def secretsdb_self_join(secretsdb, domain,
"priorChanged",
"krb5Keytab",
"privateKeytab"]
+
+ if realm is not None:
+ if dnsdomain is None:
+ dnsdomain = realm.lower()
+ dnsname = '%s.%s' % (netbiosname.lower(), dnsdomain.lower())
+ else:
+ dnsname = None
+ shortname = netbiosname.lower()
#We don't need to set msg["flatname"] here, because rdn_name will handle it, and it causes problems for modifies anyway
msg = ldb.Message(ldb.Dn(secretsdb, "flatname=%s,cn=Primary Domains" % domain))
msg["secureChannelType"] = [str(secure_channel_type)]
msg["objectClass"] = ["top", "primaryDomain"]
- if realm is not None:
- if dnsdomain is None:
- dnsdomain = realm.lower()
+ if dnsname is not None:
msg["objectClass"] = ["top", "primaryDomain", "kerberosSecret"]
msg["realm"] = [realm]
- msg["saltPrincipal"] = ["host/%s.%s@%s" % (netbiosname.lower(), dnsdomain.lower(), realm.upper())]
+ msg["saltPrincipal"] = ["host/%s@%s" % (dnsname, realm.upper())]
msg["msDS-KeyVersionNumber"] = [str(key_version_number)]
msg["privateKeytab"] = ["secrets.keytab"]
-
msg["secret"] = [machinepass]
msg["samAccountName"] = ["%s$" % netbiosname]
msg["secureChannelType"] = [str(secure_channel_type)]
@@ -742,10 +747,17 @@ def secretsdb_self_join(secretsdb, domain,
secretsdb.modify(msg)
secretsdb.rename(res[0].dn, msg.dn)
else:
+ spn = [ 'HOST/%s' % shortname ]
+ if secure_channel_type == SEC_CHAN_BDC and dnsname is not None:
+ # we are a domain controller then we add servicePrincipalName entries
+ # for the keytab code to update
+ spn.extend([ 'HOST/%s' % dnsname ])
+ msg["servicePrincipalName"] = spn
+
secretsdb.add(msg)
-def secretsdb_setup_dns(secretsdb, setup_path, private_dir,
+def secretsdb_setup_dns(secretsdb, setup_path, names, private_dir,
realm, dnsdomain,
dns_keytab_path, dnspass):
"""Add DNS specific bits to a secrets database.
@@ -764,6 +776,8 @@ def secretsdb_setup_dns(secretsdb, setup_path, private_dir,
"DNSDOMAIN": dnsdomain,
"DNS_KEYTAB": dns_keytab_path,
"DNSPASS_B64": b64encode(dnspass),
+ "HOSTNAME": names.hostname,
+ "DNSNAME" : '%s.%s' % (names.netbiosname.lower(), names.dnsdomain.lower())
})
@@ -944,6 +958,8 @@ def setup_self_join(samdb, names,
"DNSDOMAIN": names.dnsdomain,
"DOMAINDN": names.domaindn,
"DNSPASS_B64": b64encode(dnspass),
+ "HOSTNAME" : names.hostname,
+ "DNSNAME" : '%s.%s' % (names.netbiosname.lower(), names.dnsdomain.lower())
})
def getpolicypath(sysvolpath, dnsdomain, guid):
@@ -1583,7 +1599,7 @@ def provision(setup_dir, logger, session_info,
if serverrole == "domain controller":
- secretsdb_setup_dns(secrets_ldb, setup_path,
+ secretsdb_setup_dns(secrets_ldb, setup_path, names,
paths.private_dir,
realm=names.realm, dnsdomain=names.dnsdomain,
dns_keytab_path=paths.dns_keytab,
diff --git a/source4/scripting/python/samba/upgradehelpers.py b/source4/scripting/python/samba/upgradehelpers.py
index 0a896d8..d8474de 100755
--- a/source4/scripting/python/samba/upgradehelpers.py
+++ b/source4/scripting/python/samba/upgradehelpers.py
@@ -374,7 +374,7 @@ def newprovision(names, setup_dir, creds, session, smbconf, provdir, logger):
domain=names.domain, domainguid=names.domainguid,
domainsid=str(names.domainsid), ntdsguid=names.ntdsguid,
policyguid=names.policyid, policyguid_dc=names.policyid_dc,
- hostname=names.netbiosname, hostip=None, hostip6=None,
+ hostname=names.netbiosname.lower(), hostip=None, hostip6=None,
invocationid=names.invocation, adminpass=names.adminpass,
krbtgtpass=None, machinepass=None, dnspass=None, root=None,
nobody=None, wheel=None, users=None,
diff --git a/source4/setup/provision_dns_add.ldif b/source4/setup/provision_dns_add.ldif
index ac818a5..a0a8187 100644
--- a/source4/setup/provision_dns_add.ldif
+++ b/source4/setup/provision_dns_add.ldif
@@ -88,15 +88,19 @@ dnsRecord:: BAABAAUIAAAAAAAAAAAAAAAAAAAAAAAAwDqAHg==
# NOTE: This account is SAMBA4 specific!
-dn: CN=dns,CN=Users,${DOMAINDN}
+# we have it to avoid the need for the bind daemon to
+# have access to the whole secrets.keytab for the domain,
+# otherwise bind could impersonate any user
+dn: CN=dns-${HOSTNAME},CN=Users,${DOMAINDN}
objectClass: top
objectClass: person
objectClass: organizationalPerson
--
Samba Shared Repository
More information about the samba-cvs
mailing list