ntlm_auth autobuild failures on the Catalyst Cloud after XFILE

Fri Jan 6 08:33:50 UTC 2017

(resend from correct account)

I'm sorry to add to this debate around flapping tests, but at the same
time I don't want to miss out a possibly important data point.

Since around the time of the XFILE -> FILE conversion (13 Dec), the
builds that my team at Catalyst have been doing on the Catalyst cloud
have been failing with the attached.  (We also see the ctdb errors that
martin has been fixing).

These builds are on Ubuntu 14.04 x86_64, so I'm surprised that it works
on the sn-devel host, but as we know our test suite is a fickle thing. 

Attached is the output from autobuild.  

(If it doesn't reproduce locally, then access to such cloud hosts can
be arranged for team members.)

Any assistance understanding what is going on would be most valued. 

Perhaps it is a semantics difference around (flushing) FILE buffers?

Thanks,

Andrew Bartlett

-------- Forwarded Message --------
From: samba-dev at catalyst.net.nz
To: samba-dev at catalyst.net.nz
Subject: autobuild[master] failure on samba-build-multi-process-samba-
ad-dc-for-review-abartlet-0063 for task samba during test
Date: Tue, 13 Dec 2016 02:49:33 +0000

Dear Developer,

Your autobuild on samba-build-multi-process-samba-ad-dc-for-review-
abartlet-0063 failed after 198.7 minutes
when trying to test samba with the following error:

   samba: [test] failed 'make test FAIL_IMMEDIATELY=1' with status 2

the autobuild has been abandoned. Please fix the error and resubmit.

A summary of the autobuild process is here:

  /home/ubuntu/samba/autobuild.log

You can see logs of the failed task here:

  /home/ubuntu/samba/samba.stdout
  /home/ubuntu/samba/samba.stderr

or you can get full logs of all tasks in this job here:

  /home/ubuntu/samba/logs.tar.gz

The top commit for the tree that was built was:

commit 95cb3b72d99b77f42440036ed0f09a59e6c050ae
Author: Andrew Bartlett <abartlet at samba.org>
Date:   Tue Dec 13 09:09:11 2016 +1300

    smb.conf: Deprecate the "lsa over netlogon" option

    This option will be removed soon.  It was added to ensure we did
not break networks while
    allowing the NETLOGON server to become multi-process.  We are not
aware of a client that needs this
    so the option is now flagged as deprectated to be removed in a
future release.

    Signed-off-by: Andrew Bartlett <abartlet at samba.org>

The last 50 lines of log messages:

[1788(11114)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--trusted-domains
[1789(11115)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--domain-info=BUILTIN
[1790(11116)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--domain-info=$DOMAIN
[1791(11117)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--online-status
[1792(11118)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--online-status --domain=BUILTIN
[1793(11119)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--online-status --domain=$DOMAIN
[1794(11120)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--check-secret --domain=$DOMAIN
[1795(11121)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--change-secret --domain=$DOMAIN
[1796(11122)/2025 at 2h28m45s] samba.wbinfo_simple.(ad_dc:local).
--check-secret --domain=$DOMAIN
[1797(11123)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--online-status --domain=$DOMAIN
[1798(11124)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--domain-users
[1799(11125)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--domain-groups
[1800(11126)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--name-to-sid=$DC_USERNAME
[1801(11127)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--name-to-sid=$DOMAIN/$DC_USERNAME
[1802(11128)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--user-info=$DOMAIN/$DC_USERNAME
[1803(11129)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--user-groups=$DOMAIN/$DC_USERNAME
[1804(11130)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--authenticate=$DOMAIN/$DC_USERNAME%$DC_PASSWORD
[1805(11131)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--allocate-uid
[1806(11132)/2025 at 2h28m46s] samba.wbinfo_simple.(ad_dc:local).
--allocate-gid
[1807(11133)/2025 at 2h28m46s] samba.wbinfo_sids2xids.(ad_dc:local)
[1808(11134)/2025 at 2h28m47s] samba.ntlm_auth.diagnostics(ad_dc:local)
[1809(11135)/2025 at 2h28m47s] samba.ntlm_auth.(ad_dc:local)
UNEXPECTED(failure): samba.ntlm_auth.(ad_dc:local).ntlm_auth with
NTLMSSP gss-spnego-client and gss-spnego server against winbind with
require-membership-of(ad_dc:local)
REASON: Exception: Exception: Could not determine hostname for target
computer, cannot use kerberos
Traceback (most recent call last):
  File
"/home/ubuntu/autobuild/b15247/samba/source3/torture/test_ntlm_auth.py"
, line 329, in <module>
    main()
  File
"/home/ubuntu/autobuild/b15247/samba/source3/torture/test_ntlm_auth.py"
, line 277, in main
    buf = readLine(client_in)
  File
"/home/ubuntu/autobuild/b15247/samba/source3/torture/test_ntlm_auth.py"
, line 43, in readLine
    raise ReadChildError()
__main__.ReadChildError

FAILED (1 failures, 0 errors and 0 unexpected successes in 0
testsuites)

A summary with detailed information can be found in:
  ./bin/ab/summary
TOP 10 slowest tests
samba.tests.docs -> 718
samba3.local.nss(ad_dc:local) -> 546
samba4.blackbox.upgradeprovision.alpha13 -> 526
samba4.blackbox.dbcheck.release-4-1-0rc3 -> 293
samba4.blackbox.upgradeprovision.release-4-0-0 -> 263
samba4.blackbox.dbcheck.release-4-0-0 -> 233
samba4.rpc.samr.passwords.lockout on ncacn_np with (ad_dc_ntvfs) -> 225
samba4.blackbox.dbcheck.alpha13 -> 218
samba4.blackbox.dbcheck.release-4-1-6-partial-object -> 212
samba4.rpc.samr.passwords.pwdlastset on ncacn_np with (ad_dc_ntvfs) ->
200
test: running (/usr/bin/perl
/home/ubuntu/autobuild/b15247/samba/selftest/selftest.pl --
target=samba 
--prefix=./bin/ab --srcdir=/home/ubuntu/autobuild/b15247/samba --
exclude=/home/ubuntu/autobuild/b15247/samba/selftest/skip --
testlist="/usr/bin/python
/home/ubuntu/autobuild/b15247/samba/selftest/tests.py|" --
testlist="/usr/bin/python
/home/ubuntu/autobuild/b15247/samba/source3/selftest/tests.py|" --
testlist="/usr/bin/python
/home/ubuntu/autobuild/b15247/samba/source4/selftest/tests.py|"  
--exclude=/home/ubuntu/autobuild/b15247/samba/selftest/slow --
nss_wrapper_so_path=/home/ubuntu/autobuild/b15247/samba/bin/default/lib
/nss_wrapper/libnss-wrapper.so --
resolv_wrapper_so_path=/home/ubuntu/autobuild/b15247/samba/bin/default/
lib/resolv_wrapper/libresolv-wrapper.so --
socket_wrapper_so_path=/home/ubuntu/autobuild/b15247/samba/bin/default/
lib/socket_wrapper/libsocket-wrapper.so --
uid_wrapper_so_path=/home/ubuntu/autobuild/b15247/samba/bin/default/lib
/uid_wrapper/libuid-wrapper.so --use-dns-f
 aking --socket-wrapper  && touch ./bin/ab/st_done) | /usr/bin/python
-u /home/ubuntu/autobuild/b15247/samba/selftest/filter-subunit --
expected-
failures=/home/ubuntu/autobuild/b15247/samba/selftest/knownfail --
flapping=/home/ubuntu/autobuild/b15247/samba/selftest/flapping --fail-
immediately | tee ./bin/ab/subunit | /usr/bin/python -u
/home/ubuntu/autobuild/b15247/samba/selftest/format-subunit --
prefix=./bin/ab --immediate
ERROR: test failed with exit code 1

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logs.tar.gz
Type: application/gzip
Size: 804750 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170106/59e5a058/logs.tar-0001.gz>