[SCM] Samba Shared Repository - branch master updated
Jelmer Vernooij
jelmer at samba.org
Mon Dec 5 16:44:03 MST 2011
The branch, master has been updated
via 116a150 knownfail: Mark some ldap.acl search tests as no longer failing.
via d9459ed knownfail: Remove working winbind tests.
via bf2b801 knownfail: The resolve async tests work now.
via b3f81c6 knownfail: remove now succeeding local.iconv test.
via a3949f4 knownfail: remove base.maximum_allowed test, as it appears to be passing now.
via d9aeb2f knownfail: re-add createx test.
via 512d114 knownfail: Exclude specific list of SMB2 oplock tests.
via 9968fab knownfail: Remove smb2.compound tests, they pass now.
via be37328 Remove samba4.raw.oplock.brl4. Bug 7928 has been fixed.
via 0190a55 knownfail: Match on end-of-file, not end-of-file-access.
via f186425 knownfail: samba4.base.createx_access.createx_access works.
via 850c838 knownfail: LogonControl2Ex works.
via 4c3696c knownfail: Ignore failures for drsuapi tests over other transports as well.
via 4d551a9 samba4: Remove rpc-handles mixed-shared tests from known failing list.
via 4673559 selftest: Re-mark some drsuapi tests as known failing.
via 41921d6 wafsamba: Cope with not everything having a final_libs attribute.
via f9091d6 selftest: Don't mark drsuapi tests as knownfailing.
via bef27a9 wafsamba: Use final_libs list to determine if a target needs private libraries, rather than direct dependencies list.
via 8175527 Fix typo.
via 4f03164 selftest: 'samba4.rpc.netlogon on ncalrpc with seal,padcheck.netlogon.DsrEnumerateDomainTrusts' now succeeds, remove it from the knownfail list.
via f528733 GetForestTrustInformation is implemented, remove it from knownfail.
via 112d072 rap netsessiongetinfo apparently *is* provided by samba3, so don't list it in knownfail.
via e6b4a74 s3-selftest: Don't mark samba3.blackbox.failure.success test as known failing.
via be5d9c8 filter-subunit: Fix default for flapping setting.
via 0fee2c0 selftest: Display unexpected successes and expected failures.
via e626545 selftest: Add --flapping argument to filter-subunit.
via 5643d37 selftest: Split out flapping tests into a separate file.
from 5bfd625 s4-drs: do not try to contact for replication servers that are not anymore in reps*
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 116a150231d395a10233dcce4af814659322e40c
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 23:08:10 2011 +0100
knownfail: Mark some ldap.acl search tests as no longer failing.
Autobuild-User: Jelmer Vernooij <jelmer at samba.org>
Autobuild-Date: Tue Dec 6 00:43:40 CET 2011 on sn-devel-104
commit d9459ed9d358e05bddc912eb087dcad5b9de40f8
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 21:26:28 2011 +0100
knownfail: Remove working winbind tests.
commit bf2b801a55041a10d5c5c3e75cde86f6b7f62f75
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 19:11:37 2011 +0100
knownfail: The resolve async tests work now.
commit b3f81c671a8db33000b932a782ae18899758f54f
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 18:09:05 2011 +0100
knownfail: remove now succeeding local.iconv test.
commit a3949f4c866707a1934c72573005d5e728baae3c
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 16:54:37 2011 +0100
knownfail: remove base.maximum_allowed test, as it appears to be passing now.
commit d9aeb2fc33c396906676a9bf42c34beaca6bd9b8
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 15:29:53 2011 +0100
knownfail: re-add createx test.
commit 512d11492fd751ad59b0cf3e37e0e4cf7b9f3530
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 14:27:40 2011 +0100
knownfail: Exclude specific list of SMB2 oplock tests.
commit 9968fabde3eecf5778f9494a75ff6183a6b8c463
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 02:59:44 2011 +0100
knownfail: Remove smb2.compound tests, they pass now.
commit be373289db1872761d5272e68034215e240dde83
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 02:00:34 2011 +0100
Remove samba4.raw.oplock.brl4. Bug 7928 has been fixed.
commit 0190a5589c9032f2f98e468259ac3acf1d881855
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Mon Dec 5 00:36:29 2011 +0100
knownfail: Match on end-of-file, not end-of-file-access.
commit f186425a1be9d5b5fa78e39c99dae52a44d03ed3
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 22:42:22 2011 +0100
knownfail: samba4.base.createx_access.createx_access works.
commit 850c83821c2e332c9f8c8992d3a01af23e2a42fc
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 21:40:51 2011 +0100
knownfail: LogonControl2Ex works.
commit 4c3696c02ca09b444e9b22db6ed2f8d9232dfd0f
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 21:00:43 2011 +0100
knownfail: Ignore failures for drsuapi tests over other transports as well.
commit 4d551a91a5098a48b7fce7fb7091b4d573436f72
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 20:23:29 2011 +0100
samba4: Remove rpc-handles mixed-shared tests from known failing list.
commit 4673559894d155c7525c1cc58c930051185649d3
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 16:45:58 2011 +0100
selftest: Re-mark some drsuapi tests as known failing.
commit 41921d66825f9e09168a53b38afa949c684c552b
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 15:36:27 2011 +0100
wafsamba: Cope with not everything having a final_libs attribute.
commit f9091d6fa048d110666fa04fcc9b70725314f111
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 14:59:07 2011 +0100
selftest: Don't mark drsuapi tests as knownfailing.
commit bef27a98900fcef17929587264e3859a0d5c7f05
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 14:58:11 2011 +0100
wafsamba: Use final_libs list to determine if a target needs private libraries, rather than direct dependencies list.
commit 817552779d699f1242fea17dc8ae4283949036ed
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 14:56:13 2011 +0100
Fix typo.
commit 4f031643731b3cc2682516ecf82a6f682dffe8ff
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 14:22:40 2011 +0100
selftest: 'samba4.rpc.netlogon on ncalrpc with seal,padcheck.netlogon.DsrEnumerateDomainTrusts' now succeeds, remove it from the knownfail list.
commit f528733d7b241d3536abdb7ae9c7aad5e7ea1bc4
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 05:36:08 2011 +0100
GetForestTrustInformation is implemented, remove it from knownfail.
commit 112d07256a795f5774d1f1a8138b35269755cd37
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 05:06:15 2011 +0100
rap netsessiongetinfo apparently *is* provided by samba3, so don't list it in knownfail.
commit e6b4a7440639df21c7d4762369fab0ee57295c59
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 04:38:07 2011 +0100
s3-selftest: Don't mark samba3.blackbox.failure.success test as known failing.
commit be5d9c83e38fdc1d22524551eab4d4c48d7c4872
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 03:55:11 2011 +0100
filter-subunit: Fix default for flapping setting.
commit 0fee2c01b8d33710d63bb697351a6a414100b5c2
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 01:55:23 2011 +0100
selftest: Display unexpected successes and expected failures.
commit e62654578b3aa1ddafd8faf85b65d9702f49fba3
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 00:23:02 2011 +0100
selftest: Add --flapping argument to filter-subunit.
commit 5643d37a000971070b22af7c2710f66b49b57eff
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Sun Dec 4 00:09:04 2011 +0100
selftest: Split out flapping tests into a separate file.
-----------------------------------------------------------------------
Summary of changes:
buildtools/wafsamba/samba_utils.py | 2 +-
selftest/filter-subunit | 34 +++++++++++++++---
selftest/flapping | 19 ++++++++++
selftest/knownfail | 61 +++++++++++++++-----------------
selftest/subunithelper.py | 68 ++++++++++++++++++++++++++++++------
selftest/wscript | 2 +-
source3/selftest/s3-selftest.sh | 2 +-
7 files changed, 137 insertions(+), 51 deletions(-)
create mode 100644 selftest/flapping
Changeset truncated at 500 lines:
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index 5e16a5d..71cfbc5 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -65,7 +65,7 @@ def ADD_LD_LIBRARY_PATH(path):
def needs_private_lib(bld, target):
'''return True if a target links to a private library'''
- for lib in getattr(target, "uselib_local", []):
+ for lib in getattr(target, "final_libs", []):
t = bld.name_to_obj(lib, bld.env)
if t and getattr(t, 'private_library', False):
return True
diff --git a/selftest/filter-subunit b/selftest/filter-subunit
index 5f062d1..bced14b 100755
--- a/selftest/filter-subunit
+++ b/selftest/filter-subunit
@@ -1,7 +1,22 @@
#!/usr/bin/env python
# Filter a subunit stream
-# Copyright (C) Jelmer Vernooij <jelmer at samba.org>
-# Published under the GNU GPL, v3 or later
+# Copyright (C) 2009-2011 Jelmer Vernooij <jelmer at samba.org>
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# NOTE: This script is a hack, meant as a placeholder until we can migrate
+# to upstream subunit's filtering tools.
import optparse
import os
@@ -17,9 +32,11 @@ parser = optparse.OptionParser("filter-subunit [options] < instream > outstream"
parser.add_option("--expected-failures", type="string",
help="File containing list of regexes matching tests to consider known "
"failures")
-parser.add_option("--strip-passed-output", action="store_true",
+parser.add_option("--flapping", type="string",
+ help="File containing list of flapping tests, of which to ignore results.")
+parser.add_option("--strip-passed-output", action="store_true",
help="Whether to strip output from tests that passed")
-parser.add_option("--fail-immediately", action="store_true",
+parser.add_option("--fail-immediately", action="store_true",
help="Whether to stop on the first error", default=False)
parser.add_option("--prefix", type="string",
help="Add prefix to all test names")
@@ -42,6 +59,12 @@ if opts.expected_failures:
else:
expected_failures = {}
+
+if opts.flapping:
+ flapping = subunithelper.read_test_regexes(opts.flapping)
+else:
+ flapping = {}
+
statistics = {
'TESTS_UNEXPECTED_OK': 0,
'TESTS_EXPECTED_OK': 0,
@@ -58,7 +81,8 @@ signal.signal(signal.SIGINT, handle_sigint)
out = subunithelper.SubunitOps(sys.stdout)
msg_ops = subunithelper.FilterOps(out, opts.prefix, expected_failures,
opts.strip_passed_output,
- fail_immediately=opts.fail_immediately)
+ fail_immediately=opts.fail_immediately,
+ flapping=flapping)
try:
ret = subunithelper.parse_results(msg_ops, statistics, sys.stdin)
diff --git a/selftest/flapping b/selftest/flapping
new file mode 100644
index 0000000..e8a6b49
--- /dev/null
+++ b/selftest/flapping
@@ -0,0 +1,19 @@
+# This file contains a list of regular expressions matching the names of
+# tests that are flapping. In other words, they sometimes succeed and
+# sometimes fail, depending on external factors.
+#
+# "make test" will not report failures or successes for tests listed here.
+#
+# DO NOT ADD TESTS HERE UNLESS THEY ARE ACTUALLY FLAPPING
+#
+# It is much better to add known failing tests to 'knownfail', so the
+# test system can warn when they actually start passing.
+^samba3.raw.mux.* #This test is flaky on the async lock time
+^samba3.smbtorture_s3.*OPLOCK4 # fails sometimes on sn-devel
+^samba4.nbt.winsreplication.owned # fails sometimes, timing related
+^samba3.posix_s3.rpc.spoolss.*printserver.enum_printers_old # fails on some hosts due to timing issues ?
+^samba3.posix_s3.rpc.spoolss.printer.*addprinterex.print_test # another intermittent failure
+.*printer.*print_test_extended # fails on some hosts due to timing issues ?
+.*printer.*print_test # fails on some hosts due to timing issues ?
+^samba3.posix_s3.rap.printing # fails sometimes on sn-devel
+^samba3.posix_s3.smb2.lock.*.rw-exclusive # another intermittent failure
diff --git a/selftest/knownfail b/selftest/knownfail
index 6f83881..754ffb3 100644
--- a/selftest/knownfail
+++ b/selftest/knownfail
@@ -4,25 +4,14 @@
# "make test" will not report failures for tests listed here and will consider
# a successful run for any of these tests an error.
-^samba3.blackbox.failure # this is designed to fail, for testing our test infrastructure
-.*printer.*print_test_extended # fails on some hosts due to timing issues ?
-.*printer.*print_test # fails on some hosts due to timing issues ?
-^samba3.posix_s3.rap.printing # fails sometimes on sn-devel
-^samba3.posix_s3.rpc.spoolss.*printserver.enum_printers_old # fails on some hosts due to timing issues ?
-^samba3.posix_s3.rpc.spoolss.printer.*addprinterex.print_test # another intermittent failure
-^samba3.posix_s3.smb2.lock.*.rw-exclusive # another intermittent failure
+^samba3.blackbox.failure.failure # this is designed to fail, for testing our test infrastructure
.*driver.add_driver_timestamps # we only can store dates, not timestamps
-^samba3.raw.mux.* #This test is flaky on the async lock time
-^samba3.smbtorture_s3.*OPLOCK4 # fails sometimes on sn-devel
^samba3.smbtorture_s3.LOCAL-MEMCACHE #fails
^samba3.smbtorture_s3.LOCAL-TALLOC-DICT #fails
^samba3.posix_s3.nbt.dgram.*netlogon2
^samba3.*rap.sam.*.useradd # Not provided by Samba 3
^samba3.*rap.sam.*.userdelete # Not provided by Samba 3
-^samba3.*rap.basic.*.netsessiongetinfo # Not provided by Samba 3
^samba3.posix_s3.libsmbclient .opendir # This requires a workgroup called 'WORKGROUP' and for netbios browse lists to have been registered
-^samba4.local.resolve.*.async
-^samba4.local.iconv.*.next_codepoint()
# these show that we still have some differences between our system
# with our internal iconv because it passes except when we bypass our
# internal iconv modules
@@ -38,25 +27,21 @@
^samba4.local.registry.local.security
^samba4.rpc.wkssvc
^samba4.rpc.handles.*.lsarpc-shared
-^samba4.rpc.handles.*.mixed-shared
^samba4.rpc.epmapper
-^samba4.rpc.drsuapi.*
^samba4.rpc.lsalookup
^samba4.rpc.cracknames
^samba4.rpc.netlogon.*.LogonUasLogon
^samba4.rpc.netlogon.*.LogonUasLogoff
^samba4.rpc.netlogon.*.DatabaseSync
^samba4.rpc.netlogon.*.DatabaseSync2
-^samba4.rpc.netlogon.*.LogonControl
-^samba4.rpc.netlogon.*.LogonControl2
-^samba4.rpc.netlogon.*.DsrEnumerateDomainTrusts
+^samba4.rpc.netlogon.*.LogonControl$
+^samba4.rpc.netlogon.*.LogonControl2$
^samba4.rpc.netlogon.*.NetrEnumerateTrustedDomains
^samba4.rpc.netlogon.*.NetrEnumerateTrustedDomainsEx
^samba4.rpc.netlogon.*.GetPassword
^samba4.rpc.netlogon.*.GetTrustPasswords
^samba4.rpc.netlogon.*.DatabaseRedo
^samba4.rpc.netlogon.*.ServerGetTrustInfo
-^samba4.rpc.netlogon.*.GetForestTrustInformation
^samba4.rpc.samr.passwords.badpwdcount # Not provided by Samba 4 yet
^samba4.rpc.samr.passwords.lockout
^samba4.base.charset.*.Testing partial surrogate
@@ -67,10 +52,6 @@
^samba4.rap.*netremotetod
^samba4.smb2.persistent.handles1
^samba4.winbind.struct.*.show_sequence # Not yet working in winbind
-^samba4.winbind.struct.*.getpwent # Not yet working in winbind
-^samba4.winbind.struct.*.setpwent # Not yet working in winbind
-^samba4.winbind.struct.*.lookup_name_sid # Not yet working in winbind
-^samba4.winbind.struct.*.list_groups
^samba4.*base.delaywrite.*update of write time and SMBwrite truncate$
^samba4.*base.delaywrite.*update of write time and SMBwrite truncate expand$
^samba4.*base.delaywrite.*delayed update of write time 3a$
@@ -80,14 +61,10 @@
^samba4.ldap.python \(dc\).Test add_ldif\(\) with BASE64 security descriptor input using WRONG domain SID$
# some operations don't work over the CIFS NTVFS backend yet (eg. root_fid)
^samba4.ntvfs.cifs.*.base.createx_sharemodes_dir
-^samba4.ntvfs.cifs.*.base.maximum_allowed
-^samba4.*.base.createx_access # this test is broken for non-administrator users
-^samba4.smb2.oplock # oplocks in the s4 SMB2 server are a mess
^samba4.raw.lock.*.async # bug 6960
^samba4.smb2.lock.*.multiple-unlock # bug 6959
-^samba4.raw.sfileinfo.*.end-of-file # bug 6962
+^samba4.raw.sfileinfo.*.end-of-file$ # bug 6962
^samba4.raw.oplock.*.batch22 # bug 6963
-^samba4.raw.oplock.*.brl4 # bug 7928
^samba4.raw.lock.*.zerobyteread # bug 6974
^samba4.smb2.lock.*.zerobyteread # bug 6974
^samba4.raw.streams.*.delete
@@ -99,13 +76,33 @@
^samba4.smb2.acls.*.generic
^samba4.smb2.acls.*.inheritflags
^samba4.smb2.acls.*.owner
-^samba4.smb2.compound.*.related1
-^samba4.smb2.compound.*.related2
-^samba4.smb2.compound.*.invalid2
-^samba4.ldap.acl.*.search.* # ACL search behaviour not enabled by default
^samba4.ldap.acl.*.ntSecurityDescriptor.* # ACL extended checks on search not enabled by default
-^samba4.nbt.winsreplication.owned # fails sometimes, timing related
^samba4.ldap.dirsync.python.dc..__main__.ExtendedDirsyncTests.test_dirsync_deleted_items
#^samba4.ldap.dirsync.python.dc..__main__.ExtendedDirsyncTests.*
^samba4.drs.fsmo.python
^samba4.libsmbclient.opendir.opendir # This requires netbios browsing
+^samba4.rpc.drsuapi .*.drsuapi.DsGetDomainControllerInfo$
+^samba4.rpc.drsuapi .*.drsuapi.DsCrackNames$
+^samba4.smb2.oplock.exclusive2$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.exclusive5$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.exclusive6$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.brl3$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.levelii500$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.brl1$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch22$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch19$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch12$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch11$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch1$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch6$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch9$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch10$ # samba 4 oplocks are a mess
+^samba4.smb2.oplock.batch20$ # samba 4 oplocks are a mess
+^samba4.ntvfs.cifs.krb5.base.createx_access.createx_access$
+^samba4.ldap.acl.*.AclSearchTests.test_search_anonymous3$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search1$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search2$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search3$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search4$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search5$ # ACL search behaviour not enabled by default
+^samba4.ldap.acl.*.AclSearchTests.test_search6$ # ACL search behaviour not enabled by default
diff --git a/selftest/subunithelper.py b/selftest/subunithelper.py
index c59b6d0..b498878 100644
--- a/selftest/subunithelper.py
+++ b/selftest/subunithelper.py
@@ -22,8 +22,9 @@ import sys
import subunit
import subunit.iso8601
import testtools
+from testtools import content, content_type
-VALID_RESULTS = ['success', 'successful', 'failure', 'fail', 'skip', 'knownfail', 'error', 'xfail', 'skip-testsuite', 'testsuite-failure', 'testsuite-xfail', 'testsuite-success', 'testsuite-error']
+VALID_RESULTS = ['success', 'successful', 'failure', 'fail', 'skip', 'knownfail', 'error', 'xfail', 'skip-testsuite', 'testsuite-failure', 'testsuite-xfail', 'testsuite-success', 'testsuite-error', 'uxsuccess']
class TestsuiteEnabledTestResult(testtools.testresult.TestResult):
@@ -33,7 +34,6 @@ class TestsuiteEnabledTestResult(testtools.testresult.TestResult):
def parse_results(msg_ops, statistics, fh):
exitcode = 0
- expected_fail = 0
open_tests = {}
while fh:
@@ -111,7 +111,17 @@ def parse_results(msg_ops, statistics, fh):
else:
statistics['TESTS_EXPECTED_FAIL']+=1
msg_ops.addExpectedFailure(test, remote_error)
- expected_fail+=1
+ elif result in ("uxsuccess", ):
+ try:
+ test = open_tests.pop(testname)
+ except KeyError:
+ statistics['TESTS_ERROR']+=1
+ exitcode = 1
+ msg_ops.addError(subunit.RemotedTestCase(testname), subunit.RemoteError(u"Test was never started"))
+ else:
+ statistics['TESTS_UNEXPECTED_OK']+=1
+ msg_ops.addUnexpectedSuccess(test, remote_error)
+ exitcode = 1
elif result in ("failure", "fail"):
try:
test = open_tests.pop(testname)
@@ -285,9 +295,16 @@ class FilterOps(testtools.testresult.TestResult):
self._ops.addExpectedFailure(test, details)
self.output = None
+ def addUnexpectedSuccess(self, test, details=None):
+ test = self._add_prefix(test)
+ self._ops.addUnexpectedSuccess(test, details)
+ self.output = None
+
def addFailure(self, test, details=None):
test = self._add_prefix(test)
xfail_reason = find_in_list(self.expected_failures, test.id())
+ if xfail_reason is None:
+ xfail_reason = find_in_list(self.flapping, test.id())
if xfail_reason is not None:
self.xfail_added+=1
self.total_xfail+=1
@@ -308,7 +325,22 @@ class FilterOps(testtools.testresult.TestResult):
def addSuccess(self, test, details=None):
test = self._add_prefix(test)
- self._ops.addSuccess(test, details)
+ xfail_reason = find_in_list(self.expected_failures, test.id())
+ if xfail_reason is not None:
+ self.uxsuccess_added += 1
+ self.total_uxsuccess += 1
+ if details is None:
+ details = {}
+ details['reason'] = content.Content(
+ content_type.ContentType("text", "plain",
+ {"charset": "utf8"}), lambda: xfail_reason)
+ self._ops.addUnexpectedSuccess(test, details)
+ if self.output:
+ self._ops.output_msg(self.output)
+ if self.fail_immediately:
+ raise ImmediateFail()
+ else:
+ self._ops.addSuccess(test, details)
self.output = None
def skip_testsuite(self, name, reason=None):
@@ -319,6 +351,7 @@ class FilterOps(testtools.testresult.TestResult):
self.error_added = 0
self.fail_added = 0
self.xfail_added = 0
+ self.uxsuccess_added = 0
def end_testsuite(self, name, result, reason=None):
xfail = False
@@ -346,7 +379,8 @@ class FilterOps(testtools.testresult.TestResult):
self._ops.end_testsuite(name, result, reason)
def __init__(self, out, prefix=None, expected_failures=None,
- strip_ok_output=False, fail_immediately=False):
+ strip_ok_output=False, fail_immediately=False,
+ flapping=None):
self._ops = out
self.seen_output = False
self.output = None
@@ -355,12 +389,18 @@ class FilterOps(testtools.testresult.TestResult):
self.expected_failures = expected_failures
else:
self.expected_failures = {}
+ if flapping is not None:
+ self.flapping = flapping
+ else:
+ self.flapping = {}
self.strip_ok_output = strip_ok_output
self.xfail_added = 0
self.fail_added = 0
+ self.uxsuccess_added = 0
self.total_xfail = 0
self.total_error = 0
self.total_fail = 0
+ self.total_uxsuccess = 0
self.error_added = 0
self.fail_immediately = fail_immediately
@@ -484,10 +524,13 @@ class PlainFormatter(TestsuiteEnabledTestResult):
def addSkip(self, test, details=None):
self.end_test(test.id(), "skip", False, details)
- def addExpectedFail(self, test, details=None):
+ def addExpectedFailure(self, test, details=None):
self.end_test(test.id(), "xfail", False, details)
- def end_test(self, testname, result, unexpected, reason=None):
+ def addUnexpectedSuccess(self, test, details=None):
+ self.end_test(test.id(), "uxsuccess", True, details)
+
+ def end_test(self, testname, result, unexpected, details=None):
if not unexpected:
self.test_output[self.name] = ""
if not self.immediate:
@@ -502,17 +545,18 @@ class PlainFormatter(TestsuiteEnabledTestResult):
self.test_output[self.name] = ""
self.test_output[self.name] += "UNEXPECTED(%s): %s\n" % (result, testname)
- if reason is not None:
- self.test_output[self.name] += "REASON: %s\n" % (unicode(reason[1]).encode("utf-8").strip(),)
+ if details is not None:
+ self.test_output[self.name] += "REASON: %s\n" % (unicode(details[1]).encode("utf-8").strip(),)
if self.immediate and not self.verbose:
- print self.test_output[self.name]
+ sys.stdout.write(self.test_output[self.name])
self.test_output[self.name] = ""
if not self.immediate:
sys.stdout.write({
'error': 'E',
'failure': 'F',
+ 'uxsuccess': 'U',
'success': 'S'}.get(result, "?"))
def write_summary(self, path):
@@ -546,14 +590,16 @@ class PlainFormatter(TestsuiteEnabledTestResult):
if (not self.suitesfailed and
not self.statistics['TESTS_UNEXPECTED_FAIL'] and
+ not self.statistics['TESTS_UNEXPECTED_OK'] and
not self.statistics['TESTS_ERROR']):
ok = (self.statistics['TESTS_EXPECTED_OK'] +
self.statistics['TESTS_EXPECTED_FAIL'])
print "\nALL OK (%d tests in %d testsuites)" % (ok, self.suites_ok)
else:
- print "\nFAILED (%d failures and %d errors in %d testsuites)" % (
+ print "\nFAILED (%d failures, %d errors and %d unexpected successes in %d testsuites)" % (
self.statistics['TESTS_UNEXPECTED_FAIL'],
self.statistics['TESTS_ERROR'],
+ self.statistics['TESTS_UNEXPECTED_OK'],
len(self.suitesfailed))
def skip_testsuite(self, name, reason="UNKNOWN"):
diff --git a/selftest/wscript b/selftest/wscript
index 5a2cbd1..67ea5a4 100644
--- a/selftest/wscript
+++ b/selftest/wscript
@@ -88,7 +88,7 @@ def cmd_testonly(opt):
env.SUBUNIT_FORMATTER = os.getenv('SUBUNIT_FORMATTER')
if not env.SUBUNIT_FORMATTER:
env.SUBUNIT_FORMATTER = '${PYTHON} -u ${srcdir}/selftest/format-subunit --prefix=${SELFTEST_PREFIX} --immediate'
- env.FILTER_XFAIL = '${PYTHON} -u ${srcdir}/selftest/filter-subunit --expected-failures=${srcdir}/selftest/knownfail'
+ env.FILTER_XFAIL = '${PYTHON} -u ${srcdir}/selftest/filter-subunit --expected-failures=${srcdir}/selftest/knownfail --flapping=${srcdir}/selftest/flapping'
if Options.options.FAIL_IMMEDIATELY:
env.FILTER_XFAIL += ' --fail-immediately'
diff --git a/source3/selftest/s3-selftest.sh b/source3/selftest/s3-selftest.sh
index 4943b1e..0d20324 100755
--- a/source3/selftest/s3-selftest.sh
+++ b/source3/selftest/s3-selftest.sh
@@ -1,6 +1,6 @@
#!/bin/sh
-FILTER_XFAIL="${PYTHON} -u ${SELFTESTDIR}/filter-subunit --expected-failures=${SELFTESTDIR}/knownfail"
+FILTER_XFAIL="${PYTHON} -u ${SELFTESTDIR}/filter-subunit --expected-failures=${SELFTESTDIR}/knownfail --flapping=${SELFTESTDIR}/flapping"
if [ "x${SUBUNIT_FORMATTER}" = x"" ]; then
SUBUNIT_FORMATTER="${PYTHON} -u ${SELFTESTDIR}/format-subunit --prefix=${SELFTESTPREFIX} --immediate"
fi
--
Samba Shared Repository
More information about the samba-cvs
mailing list