flapping winbind test: LDAP server stuck? Re: autobuild[sn-devel-144]: intermittent test failure detected

[Andrew Bartlett]

On Mon, 2017-01-09 at 03:38 +0100, autobuild wrote:
> The autobuild test system (on sn-devel-144) has detected an
> intermittent failing test in 
> the current master tree.
> 
> The autobuild log of the failure is available here:
> 
>    http://git.samba.org/autobuild.flakey.sn-devel-144/2017-01-09-0338
> /flakey.log
> 
> The samba build logs are available here:
> 
>    http://git.samba.org/autobuild.flakey.sn-devel-144/2017-01-09-0338
> /samba.stderr
>    http://git.samba.org/autobuild.flakey.sn-devel-144/2017-01-09-0338
> /samba.stdout

I find this really curious:

server process 654063 took more than 31 seconds to exit, sending
SIGTERM
Exiting pid 654063 on SIGTERM
samba child process 654063 exited with value 127
SAMBA LOG of: S4MEMBER pid 654063
/memdisk/autobuild/fl/b169311/samba/bin/winbindd: Reducing LDAP page
size from 1000 to 500 due to IO_TIMEOUT
/memdisk/autobuild/fl/b169311/samba/bin/winbindd:
kerberos_kinit_password S4MEMBER$@SAMBA.EXAMPLE.COM failed:
Preauthentication failed
/memdisk/autobuild/fl/b169311/samba/bin/winbindd: ads_search_retry:
failed to reconnect (Preauthentication failed)
Exiting pid 654063 on SIGTERM

Even more curious is that the backtrace shows it in epoll despite
closing of stdin not shutting down the process.  However it suggests to
me that the AD DC might be stuck somehow for us to have hit the
IO_TIMEOUT in the winbindd code. 

Also in that log is:

ldb: descriptor_modify: Could not find SD for OU=Original
parent,DC=samba,DC=example,DC=com

../source4/dsdb/repl/replicated_objects.c:933 Failed to prepare commit
of transaction: descriptor_modify on OU=Test Child 2,OU=Original
parent,DC=samba,DC=example,DC=com failed: No such Base DN: OU=Original
parent,DC=samba,DC=example,DC=com
Failed to commit objects:
WERR_GEN_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE

However I don't know if this situation is the one that has been
deliberately engineered or not.

I suggest keeping an open mind on this one, it might be a hairy one. 
I'll let the list know if I find anything.

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba