Samba4 / OpenLDAP testing
Howard Chu
hyc at highlandsun.com
Sat Jun 6 09:12:37 GMT 2009
With a current git pull and the OpenLDAP 2.4.17 release candidate "make test"
gives me 5 failures and 0 errors in 8 testsuites. A couple of those are
apparently schema issues, some ops are rejected by slapd due to syntax violations.
There are also some problems with the slapd.conf being used in the tests; it's
forcing slapd to scan the entire database multiple times because a lot of the
attributes being searched on are unindexed. This is compounded by the use of
the refint overlay, which is currently configured with 68 attributes (and each
of those invokes a search, which is also unindexed). In fact, these 68
attributes are pairs of forward/backward links that are also being maintained
by the memberof overlay, so half of those attributes (the backlinks) don't
need to be managed by refint. I.e., it's doing twice as much work as it needs to.
Also, Samba4.pm needs to do a clean shutdown of slapd instead of using kill -9
on it; otherwise the databases aren't closed and it forces a recovery on next
startup. This causes extra delay between the provisioning and actual test
phase (which is particularly painful when you're using valgrind or some other
malloc debugger on slapd...).
The attached Samba4.pm.txt diff fixes the slapd shutdown issue.
The provision.py.txt diff fixes the refint configuration.
While trying to chase some other OpenLDAP issues I found it helpful to mark
where a particular test began in the slapd logs. The selftest.pl.txt diff adds
this. (It's probably a bit too stupid; it always writes the name in
dc/private/ldap/logs and for some of the tests the action is occurring in some
other server. Not sure yet whether we can reliably parse the (dc) / (member)
tags out of the test names to help it decide where to print the test name.)
Andrew suggested autoconfiguring the indexing; I probably won't have time to
write that up but will give it some thought. For the moment, here's a list of
attributes that need to be indexed; this is just pulled from "grep 'not
indexed'" from the slapd logs. In the attached slapd.conf.txt diff I only
indexed the topmost attributes, figuring the others aren't requested often
enough to justify the index write overhead. In the unindexed.txt list the
number of occurrences of unindexed search is in the first column. If an
attribute appears twice in the list that's because it occurred for both
equality and substring indexing.
I also added "dbnosync" to the configs; I test on a number of machines with
slow disks and would rather not wait for a synchronous disk op for every
directory write. I also changed the authz-regexp rules to use a simple
substitution, since that's all that this particular test configuration
requires. back-ldif is pretty slow since it opens individual filesystem
directories and files to satisfy any directory operation. I don't feel too
strongly about that change in the long run, but I prefer to run it this way
myself.
These changes allow slapd to do a lot less work, which consequently allows the
tests to complete a lot sooner.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
-------------- next part --------------
diff --git a/source4/scripting/python/samba/provision.py b/source4/scripting/python/samba/provision.py
index 189c93a..c0095ff 100644
--- a/source4/scripting/python/samba/provision.py
+++ b/source4/scripting/python/samba/provision.py
@@ -1313,7 +1313,7 @@ def provision_backend(setup_dir=None, message=None,
attribute="lDAPDisplayName",
scope=SCOPE_SUBTREE)
if target is not None:
- refint_attributes = refint_attributes + " " + target + " " + res[i]["lDAPDisplayName"][0]
+ refint_attributes = refint_attributes + " " + res[i]["lDAPDisplayName"][0]
memberof_config += read_and_sub_file(setup_path("memberof.conf"),
{ "MEMBER_ATTR" : str(res[i]["lDAPDisplayName"][0]),
-------------- next part --------------
diff --git a/selftest/selftest.pl b/selftest/selftest.pl
index 3653523..b916a6c 100755
--- a/selftest/selftest.pl
+++ b/selftest/selftest.pl
@@ -887,6 +887,11 @@ $envvarstr
next;
}
+ if (defined($ldap)) {
+ open(LF, ">>$prefix/dc/private/ldap/logs");
+ print LF "\nTEST $name\n\n";
+ close(LF);
+ }
run_testsuite($envname, $name, $cmd, $i, $suitestotal,
$msg_ops);
-------------- next part --------------
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 781c9f3..ff39665 100644
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -40,7 +40,7 @@ sub openldap_start($$$) {
$olpath = "$olroot/libexec:$olroot/sbin:";
}
$ENV{PATH} = "$olpath/usr/local/sbin:/usr/sbin:/sbin:$ENV{PATH}";
- system("slapd -d0 -f $slapd_conf -h $uri > $logs 2>&1 &");
+ open(PIPE, "| exec slapd -d0 -f $slapd_conf -h $uri >> $logs 2>&1");
$ENV{PATH} = $oldpath;
}
@@ -78,8 +78,10 @@ sub slapd_stop($$)
} elsif ($self->{ldap} eq "openldap") {
open(IN, "<$envvars->{OPENLDAP_PIDFILE}") or
die("unable to open slapd pid file: $envvars->{OPENLDAP_PIDFILE}");
- kill 9, <IN>;
+ my $pid = <IN>;
+ kill 15, $pid;
close(IN);
+ waitpid($pid, 0);
}
return 1;
}
-------------- next part --------------
5505 (servicePrincipalName)
4896 (userPrincipalName)
3348 (flatName)
648 (groupType)
352 (sAMAccountType)
324 (trustPartner)
225 (privilege)
216 (securityIdentifier)
199 (objectSid)
23 (displayName)
23 (displayName)
18 (sn)
18 (sn)
18 (samba4RDN)
18 (samba4RDN)
18 (givenName)
18 (givenName)
8 (proxyAddresses)
8 (proxyAddresses)
8 (physicalDeliveryOfficeName)
8 (physicalDeliveryOfficeName)
8 (msDS-PhoneticLastName)
8 (msDS-PhoneticLastName)
8 (msDS-PhoneticFirstName)
8 (msDS-PhoneticFirstName)
8 (msDS-PhoneticDisplayName)
8 (msDS-PhoneticDisplayName)
8 (msDS-PhoneticDepartment)
8 (msDS-PhoneticDepartment)
8 (msDS-PhoneticCompanyName)
8 (msDS-PhoneticCompanyName)
8 (msDS-AdditionalSamAccountName)
8 (msDS-AdditionalSamAccountName)
8 (legacyExchangeDN)
8 (legacyExchangeDN)
7 (sAMAccountName)
2 (userAccountControl)
-------------- next part --------------
diff --git a/source4/setup/slapd.conf b/source4/setup/slapd.conf
index 09dffbb..59248b1 100644
--- a/source4/setup/slapd.conf
+++ b/source4/setup/slapd.conf
@@ -23,11 +23,19 @@ sasl-realm ${DNSDOMAIN}
authz-regexp
uid=([^,]*),cn=([^,]*),cn=digest-md5,cn=auth
- ldap:///cn=samba??one?(cn=\$1)
+ cn=$1,cn=samba
authz-regexp
uid=([^,]*),cn=([^,]*),cn=ntlm,cn=auth
- ldap:///cn=samba??one?(cn=\$1)
+ cn=$1,cn=samba
+
+#authz-regexp
+# uid=([^,]*),cn=([^,]*),cn=digest-md5,cn=auth
+# ldap:///cn=samba??one?(cn=\$1)
+
+#authz-regexp
+# uid=([^,]*),cn=([^,]*),cn=ntlm,cn=auth
+# ldap:///cn=samba??one?(cn=\$1)
access to dn.base=""
by dn=cn=samba-admin,cn=samba manage
@@ -78,10 +86,13 @@ index objectClass eq
index samAccountName eq
index name eq
index objectCategory eq
+index objectSid eq
index lDAPDisplayName eq
index subClassOf eq
index cn eq
index entryUUID,entryCSN eq
+index servicePrincipalName eq
+dbnosync
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
@@ -111,6 +122,8 @@ index dnsRoot eq
index nETBIOSName eq
index cn eq
index entryUUID,entryCSN eq
+index servicePrincipalName eq
+dbnosync
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
@@ -129,7 +142,7 @@ suffix ${DOMAINDN}
rootdn cn=Manager,${DOMAINDN}
directory ${LDAPDIR}/db/user
index objectClass eq
-index samAccountName eq
+index samAccountName eq,sub
index name eq
index objectSid eq
index objectCategory eq
@@ -143,6 +156,16 @@ index dnsRoot eq
index nETBIOSName eq
index cn eq
index entryUUID,entryCSN eq
+index flatName eq
+index groupType eq
+index privilege eq
+index proxyAddresses eq,sub
+index sAMAccountType eq
+index securityIdentifier eq
+index trustPartner eq
+index userPrincipalName eq
+index servicePrincipalName eq
+dbnosync
#syncprov is stable in OpenLDAP 2.3, and available in 2.2.
#We need this for the contextCSN attribute and mmr.
More information about the samba-technical
mailing list