Selftest environment interdependency graph

[Ralph Böhme]

On Mon, Jan 08, 2018 at 10:19:41AM +0100, Andreas Schneider via samba-technical wrote:
> On Monday, 8 January 2018 10:12:16 CET Andrew Bartlett wrote:
> > On Mon, 2018-01-08 at 10:06 +0100, Andreas Schneider via samba-
> > 
> > technical wrote:
> > > On Monday, 8 January 2018 07:16:17 CET Jamie McClymont via samba-technical
> > > 
> > > wrote:
> > > > Hi all,
> > > > 
> > > > I've been having a look at parallelisation of the testing process. To
> > > > that end, I have, through the Perl-parsing power of regex(!), created a
> > > > graph of dependencies between envs, including the time used running
> > > > suites for each env in one run on the catalyst cloud. This is attached
> > > > as svg for your viewing pleasure.
> > > 
> > > Hi Jamie,
> > > 
> > > looking at the graph we see that some things should be changed. Metze and
> > > I
> > > looked into moving ad_member to ad_dc:
> > > 
> > > https://git.samba.org/?p=asn/
> > > samba.git;a=commitdiff;h=7f8dc6ff0662b5e3f6bfeab30139f1de37fc2edf
> > > 
> > > However we got sidetracked by other stuff. So it doesn't work yet. There
> > > are several tests failing for unknown reasons.
> > > 
> > > More WIP is here:
> > > https://git.samba.org/?p=asn/samba.git;a=shortlog;h=refs/heads/master-self
> > > test
> > Yeah, there is much still to untangle here.  The NT4 stuff should be
> > really simple to split out, but every time I try I get very odd
> > failures, like the ones Jamie already chased down, and now this one:
> > 
> > UNEXPECTED(failure): samba4.winbind.wbclient.wbcListUsers(nt4_dc:local)
> > REASON: Exception: Exception:
> > ../nsswitch/libwbclient/tests/wbclient.c:361: wbcLookupUserSids(&sid,
> > 1, &num_sids, &sids) was WBC_ERR_DOMAIN_NOT_FOUND, expected
> > WBC_ERR_SUCCESS: wbcLookupUserSids of S-1-5-21-2923062406-1030520007-
> > 827082798-1158 failed
> 
> I think this is the test we also hit last time and then got sidetracked :-(
> 
> A problem is that a lot of tests taint environments and do not clean up. 
> Leaving test users around or use the Administrator account and change that 
> than creating a temporary user :-(

I just ran into a similar issue in the nt4_member environment: setup uses net
join, not net rpc join, so the the join when attempted with ADS fails which
triggers creating of a record for the DCs *IP* in the negative conncection
cache.

This caused interesting, timing dependent failure modes later in
winbindd. There, some codepatch check the negative connection cache with the
server name (init_dc_connection), others use the the IP
(check_domain_online_handler).

Attached patch fixed the issue for me by using net rpc join in the selftest
setup. Please review & push if happy.

-slow

-- 
Ralph Boehme, Samba Team       https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
-------------- next part --------------
From a813eb40f6fb69d93c97b3798555fee35cdefa01 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Sat, 6 Jan 2018 12:27:27 +0100
Subject: [PATCH] selftest: use net rpc join when joining NT4-style domains

Otherwise net join when failing at the CLDAP ping stage will put a
negative entry for the DC in the conncache which can trigger *hard* to
debug problems later in winbindd.

Signed-off-by: Ralph Boehme <slow at samba.org>
---
 selftest/target/Samba3.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/selftest/target/Samba3.pm b/selftest/target/Samba3.pm
index 1e652d8e3f5..f5e64725acf 100755
--- a/selftest/target/Samba3.pm
+++ b/selftest/target/Samba3.pm
@@ -347,7 +347,7 @@ sub setup_nt4_member($$$)
 	my $cmd = "";
 	$cmd .= "SOCKET_WRAPPER_DEFAULT_IFACE=\"$ret->{SOCKET_WRAPPER_DEFAULT_IFACE}\" ";
 	$cmd .= "SELFTEST_WINBINDD_SOCKET_DIR=\"$ret->{SELFTEST_WINBINDD_SOCKET_DIR}\" ";
-	$cmd .= "$net join $ret->{CONFIGURATION} $nt4_dc_vars->{DOMAIN} member";
+	$cmd .= "$net rpc join $ret->{CONFIGURATION} $nt4_dc_vars->{DOMAIN} member";
 	$cmd .= " -U$nt4_dc_vars->{USERNAME}\%$nt4_dc_vars->{PASSWORD}";
 
 	if (system($cmd) != 0) {
-- 
2.13.6