Summary of flapping tests on sn-devel
Stefan Metzmacher
metze at samba.org
Fri Mar 11 13:35:47 UTC 2016
Hi,
here's an updated patchset that hopefully pass autobuild.
The indentation in the ldap.notification fix had two whitespaces too much,
metze
Am 11.03.2016 um 13:03 schrieb Stefan Metzmacher:
> Hi Douglas,
>
> thanks for the summary!
>
>> There is a little script at
>> http://git.catalyst.net.nz/gitweb?p=samba-cloud-autobuild.git;a=blob;f=parse-email/parse-autobuild-email;hb=refs/heads/heat
>> that reads a mailbox directory containing autobuild failure emails,
>> collects the logs from the server, and prints counts of the common
>> failure messages. You can restrict it to recent failures, as in this
>> example where it shows the 8 worst offenders since a month ago:
>>
>> $ ./parse-email/parse-autobuild-email --since 2016-02-11 -c 8
>> found 97 lines matching '^UNEXPECTED' in 125 files matching 'samba.stdout$'
>> 41 UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
>
> It's sad, as I spend hours on debugging this (without success),
> but I think we should mark this as flapping,
>
>> 11 UNEXPECTED(failure): samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_max_search(ad_dc_ntvfs)
>
> I just posted a fix for that.
>
>> 7 UNEXPECTED(failure): samba4.blackbox.dbcheck.release-4-1-0rc3.check_expected_after_dup_values(none)
>
> I think that's the local-attid problem andrew was about to fix, right?
>
>> 4 UNEXPECTED(failure): samba3.unix.whoami machine account.whoami(nt4_member:local)
>> 3 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (ad_member) member creds.sending a message to the remote server(ad_member)
>> 3 UNEXPECTED(failure): samba3.wbinfo_simple.(ad_member:local).--krb5auth=SAMBADOMAIN/Administrator%locDCpass1.wbinfo(ad_member:local)
>> 3 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinter.sd(ad_dc)
>> 2 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.unable_to_decrypt_secret(ad_dc_ntvfs)
>
> That's the one you (or Garming?) tried to fix a few days ago, right?
>
> I've also (hopefully) fixed the new problem with python 2.6:
> [1804(11085)/1902 at 1h57m2s]
> samba4.ldap.sort.python(ad_dc_ntvfs)(ad_dc_ntvfs)
> Traceback (most recent call last):
> File
> "/memdisk/autobuild/fl/b3372584/samba/source4/dsdb/tests/python/sort.py", line
> 8, in <module>
> from collections import Counter
> ImportError: cannot import name Counter
> command: python
> /memdisk/autobuild/fl/b3372584/samba/source4/dsdb/tests/python/sort.py
> $SERVER -U"$USERNAME%$PASSWORD" --workgroup=$DOMAIN $LOADLIST 2>&1 |
> /memdisk/autobuild/fl/b3372584/samba/selftest/filter-subunit
> --fail-on-empty --prefix="samba4.ldap.sort.python(ad_dc_ntvfs)."
> --suffix="(ad_dc_ntvfs)"
> expanded command: python
> /memdisk/autobuild/fl/b3372584/samba/source4/dsdb/tests/python/sort.py
> localdc -U"Administrator%locDCpass1" --workgroup=SAMBADOMAIN $LOADLIST
> 2>&1 | /memdisk/autobuild/fl/b3372584/samba/selftest/filter-subunit
> --fail-on-empty --prefix="samba4.ldap.sort.python(ad_dc_ntvfs)."
> --suffix="(ad_dc_ntvfs)"
> ERROR: Testsuite[samba4.ldap.sort.python(ad_dc_ntvfs)(ad_dc_ntvfs)]
> REASON: Exit code was 1
>
> See
> https://git.samba.org/autobuild.flakey.sn-devel-104/2016-03-11-1019/samba.stdout
>
> I've attached all patches including the ldap.notification change.
>
> Please have a look.
>
> Can we also get someone else to review and push these?
>
> Thanks!
> metze
>
-------------- next part --------------
From 3e05eafb96f18f64568fe91138ffd4070ca6e689 Mon Sep 17 00:00:00 2001
From: Stefan Metzmacher <metze at samba.org>
Date: Fri, 11 Mar 2016 10:16:27 +0100
Subject: [PATCH 1/3] s4:dsdb/test/notification: make test_invalid_filter more
resilient against ordering races
We saw a lot of flapping tests with:
[1793(11038)/1892 at 1h55m26s]
samba4.ldap.notification.python(ad_dc_ntvfs)(ad_dc_ntvfs)
UNEXPECTED(failure):
samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_max_search(ad_dc_ntvfs)
REASON: Exception: Exception: Traceback (most recent call last):
File
"/memdisk/autobuild/fl/b1782183/samba/source4/dsdb/tests/python/notification.py",
line 181, in test_max_search
self.assertEquals(num, ERR_TIME_LIMIT_EXCEEDED)
AssertionError: 11 != 3
Signed-off-by: Stefan Metzmacher <metze at samba.org>
---
source4/dsdb/tests/python/notification.py | 15 +++++++++++----
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/source4/dsdb/tests/python/notification.py b/source4/dsdb/tests/python/notification.py
index d799c91..afc6343 100755
--- a/source4/dsdb/tests/python/notification.py
+++ b/source4/dsdb/tests/python/notification.py
@@ -168,6 +168,8 @@ delete: otherLoginWorkstations
attrs=["name"],
controls=["notification:1"],
timeout=1)
+ num_admin_limit = 0
+ num_time_limit = 0
for i in xrange(0, max_notifications + 1):
try:
for msg in notifies[i]:
@@ -175,10 +177,15 @@ delete: otherLoginWorkstations
res = notifies[i].result()
self.fail()
except LdbError, (num, _):
- if i >= max_notifications:
- self.assertEquals(num, ERR_ADMIN_LIMIT_EXCEEDED)
- else:
- self.assertEquals(num, ERR_TIME_LIMIT_EXCEEDED)
+ if num == ERR_ADMIN_LIMIT_EXCEEDED:
+ num_admin_limit += 1
+ continue
+ if num == ERR_TIME_LIMIT_EXCEEDED:
+ num_time_limit += 1
+ continue
+ raise
+ self.assertEqual(num_admin_limit, 1)
+ self.assertEqual(num_time_limit, max_notifications)
def test_invalid_filter(self):
"""Testing invalid filters for notifications"""
--
1.9.1
From c3a3897c2e8da1a467318b833d1a20ca0027221f Mon Sep 17 00:00:00 2001
From: Stefan Metzmacher <metze at samba.org>
Date: Fri, 11 Mar 2016 10:39:13 +0100
Subject: [PATCH 2/3] s4:dsdb/test/sort: avoid 'from collections import
Counter'
This is only available in python 2.7 and >= 3.1
This should fix make test with python 2.6.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
---
source4/dsdb/tests/python/sort.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/source4/dsdb/tests/python/sort.py b/source4/dsdb/tests/python/sort.py
index c4d2c44..b7b9d83 100644
--- a/source4/dsdb/tests/python/sort.py
+++ b/source4/dsdb/tests/python/sort.py
@@ -5,7 +5,6 @@ from unicodedata import normalize
import locale
locale.setlocale(locale.LC_ALL, ('en_US', 'UTF-8'))
-from collections import Counter
import optparse
import sys
import os
@@ -191,7 +190,13 @@ class BaseSortTests(samba.tests.TestCase):
self.expected_results[k] = (fixed, list(reversed(fixed)))
for k in ('streetAddress', 'postalAddress'):
if k in self.expected_results:
- c = Counter([u[k] for u in self.users])
+ c = {}
+ for u in self.users:
+ x = u[k]
+ if x in c:
+ c[x] += 1
+ continue
+ c[x] = 1
fixed = []
for x in FIENDISH_TESTS:
fixed += [norm(x)] * c[x]
--
1.9.1
From b85a0dc87fd451b0c650cfc7fd50c4979f026dac Mon Sep 17 00:00:00 2001
From: Stefan Metzmacher <metze at samba.org>
Date: Fri, 11 Mar 2016 10:49:21 +0100
Subject: [PATCH 3/3] selftest: mark
samba4.winbind.struct.domain_info.ad_member as flapping
See https://lists.samba.org/archive/samba-technical/2016-March/112861.html
found 517 lines matching '^UNEXPECTED' in 641 files matching 'samba.stdout$'
175 UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
19 UNEXPECTED(failure): samba4.winbind.struct.domain_info(s3member:local)
12 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.server_wrap_encrypt_decrypt_wrong_key(ad_dc_ntvfs)
12 UNEXPECTED(failure): samba4.drs.delete_object.python(promoted_dc).delete_object.DrsDeleteObjectTestCase.test_ReplicateDeletedObject1(promoted_dc)
12 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.server_wrap_decrypt_wrong_r2(ad_dc_ntvfs)
11 UNEXPECTED(failure): samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_max_search(ad_dc_ntvfs)
We'll see if we also need to add
samba4.winbind.struct.domain_info.s3member
before we're able to identify and fix the problem.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
---
selftest/flapping | 1 +
1 file changed, 1 insertion(+)
diff --git a/selftest/flapping b/selftest/flapping
index 81629f9..2f0ef55 100644
--- a/selftest/flapping
+++ b/selftest/flapping
@@ -29,3 +29,4 @@
^samba4.drs.delete_object.python # flakey on sn-devel
^samba4.blackbox.samba_tool_demote # flakey on sn-devel
^samba4.smb2.create.mkdir-dup\(ad_dc_ntvfs\) # This test (for bug 11486) involves a race, not always protected against in the NTVFS file server
+^samba4.winbind.struct.domain_info.ad_member # flakey on sn-devel-104 and sn-devel-144
--
1.9.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160311/3b3c553b/signature.sig>
More information about the samba-technical
mailing list