Summary of flapping tests on sn-devel

[Douglas Bagnall]

There is a little script at
http://git.catalyst.net.nz/gitweb?p=samba-cloud-autobuild.git;a=blob;f=parse-email/parse-autobuild-email;hb=refs/heads/heat
that reads a mailbox directory containing autobuild failure emails,
collects the logs from the server, and prints counts of the common
failure messages. You can restrict it to recent failures, as in this
example where it shows the 8 worst offenders since a month ago:

$ ./parse-email/parse-autobuild-email  --since 2016-02-11 -c 8
found 97 lines matching '^UNEXPECTED' in 125 files matching 'samba.stdout$'
  41 UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
  11 UNEXPECTED(failure): samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_max_search(ad_dc_ntvfs)
   7 UNEXPECTED(failure): samba4.blackbox.dbcheck.release-4-1-0rc3.check_expected_after_dup_values(none)
   4 UNEXPECTED(failure): samba3.unix.whoami machine account.whoami(nt4_member:local)
   3 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (ad_member) member creds.sending a message to the remote server(ad_member)
   3 UNEXPECTED(failure): samba3.wbinfo_simple.(ad_member:local).--krb5auth=SAMBADOMAIN/Administrator%locDCpass1.wbinfo(ad_member:local)
   3 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinter.sd(ad_dc)
   2 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.unable_to_decrypt_secret(ad_dc_ntvfs)

That's saying there were 41 occurrances of the
"samba4.winbind.struct.domain_info(ad_member:local)" failure.

And for the full list since you subscribed to samba-cvs (for me, about
a year, results below), just run it without arguments. It may take a
while to fetch the logs.

Douglas
-------------- next part --------------
found 517 lines matching '^UNEXPECTED' in 641 files matching 'samba.stdout$'
 175 UNEXPECTED(failure): samba4.winbind.struct.domain_info(ad_member:local)
  19 UNEXPECTED(failure): samba4.winbind.struct.domain_info(s3member:local)
  12 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.server_wrap_encrypt_decrypt_wrong_key(ad_dc_ntvfs)
  12 UNEXPECTED(failure): samba4.drs.delete_object.python(promoted_dc).delete_object.DrsDeleteObjectTestCase.test_ReplicateDeletedObject1(promoted_dc)
  12 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.server_wrap_decrypt_wrong_r2(ad_dc_ntvfs)
  11 UNEXPECTED(failure): samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_max_search(ad_dc_ntvfs)
  10 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinterex.sd(ad_dc)
   9 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.sign (ad_member) member creds.creating a good symlink and deleting it by path(ad_member)
   9 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.unable_to_decrypt_secret(ad_dc_ntvfs)
   9 UNEXPECTED(failure): samba3.blackbox.dfree_quota (fileserver).Test user quota(fileserver)
   9 UNEXPECTED(failure): samba4.rpc.cracknames on ncacn_ip_tcp.cracknames.cracknames-test(ad_dc_ntvfs)
   9 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (nt4_member) member creds.sending a message to the remote server(nt4_member)
   8 UNEXPECTED(failure): samba3.smbtorture_s3.plain(ad_dc_ntvfs).FDPASS.smbtorture(ad_dc_ntvfs)
   8 UNEXPECTED(failure): samba4.rpc.drsuapi on ncalrpc with bigendian.drsuapi.DsCrackNames(ad_dc_ntvfs:local)
   8 UNEXPECTED(failure): samba.blackbox.wbinfo(rodc:local).wbinfo -U against rodc(rodc:local)
   7 UNEXPECTED(failure): samba4.drs.delete_object.python(vampire_dc).delete_object.DrsDeleteObjectTestCase.test_ReplicateDeletedObject1(vampire_dc)
   7 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (ad_member) member creds.sending a message to the remote server(ad_member)
   7 UNEXPECTED(failure): samba4.blackbox.dbcheck.release-4-1-0rc3.check_expected_after_dup_values(none)
   7 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (nt4_dc).sending a message to the remote server(nt4_dc)
   7 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinterex.sd(nt4_dc)
   7 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinter.sd(ad_dc)
   7 UNEXPECTED(failure): samba3.smb2.create.mkdir-dup(ad_dc)
   6 UNEXPECTED(failure): samba3.raw.notify.recursive(ad_dc)
   6 UNEXPECTED(failure): samba4.base.defer_open.defer_open(ad_dc_ntvfs)
   6 UNEXPECTED(uxsuccess): samba4.smb2.create.mkdir-dup(ad_dc_ntvfs)
   6 UNEXPECTED(failure): samba4.blackbox.smbclient(chgdcpass:local).Test login with --machine-pass without kerberos(chgdcpass:local)
   5 UNEXPECTED(failure): samba4.ntvfs.cifs.krb5.base.defer_open.defer_open(ad_dc_ntvfs)
   5 UNEXPECTED(failure): samba4.tombstone_reanimation.python.tombstone_reanimation.RestoreContainerObjectTestCase.test_container(fl2000dc)
   5 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinter.sd(nt4_dc)
   5 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.sign (nt4_dc).creating a good symlink and deleting it by path(nt4_dc)
   5 UNEXPECTED(failure): samba4.blackbox.kinit_trust(fl2003dc:local).Test login with user kerberos ccache(fl2003dc:local)
   5 UNEXPECTED(failure): samba4.winbind.pac.GSSAPI(ad_member:local)
   5 UNEXPECTED(failure): samba3.smb2.durable-open.open2-lease(nt4_dc)
   4 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.sign (nt4_member) member creds.creating a good symlink and deleting it by path(nt4_member)
   4 UNEXPECTED(failure): samba3.unix.whoami machine account.whoami(nt4_member:local)
   4 UNEXPECTED(failure): samba4.tombstone_reanimation.python.tombstone_reanimation.RestoreContainerObjectTestCase.test_container(fl2008r2dc)
   4 UNEXPECTED(failure): samba3.raw.notify.recursive(nt4_dc)
   4 UNEXPECTED(failure): samba4.blackbox.kinit(fl2008r2dc:local).Test login with user kerberos ccache(fl2008r2dc:local)
   3 UNEXPECTED(failure): samba4.blackbox.dbcheck(ad_dc_ntvfs).dbcheck(ad_dc_ntvfs:local)
   3 UNEXPECTED(failure): samba.blackbox.wbinfo(promoted_dc:local).wbinfo -U against promoted_dc(promoted_dc:local)
   3 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinterex.sd(plugin_s4_dc)
   3 UNEXPECTED(error): samba4.drs.repl_schema.python(promoted_dc).repl_schema.DrsReplSchemaTestCase.test_classWithCustomAttribute(promoted_dc)
   3 UNEXPECTED(failure): samba4.blackbox.kinit(fl2000dc:local).Test login with user kerberos ccache(fl2000dc:local)
   3 UNEXPECTED(failure): samba3.wbinfo_simple.(ad_member:local).--krb5auth=SAMBADOMAIN/Administrator%locDCpass1.wbinfo(ad_member:local)
   2 UNEXPECTED(failure): samba3.rpc.spoolss.printer.addprinter.sd(plugin_s4_dc)
   2 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.sign (nt4_dc).sending a message to the remote server(nt4_dc)
   2 UNEXPECTED(failure): samba4.tombstone_reanimation.python.tombstone_reanimation.RestoreContainerObjectTestCase.test_container(fl2003dc)
   2 UNEXPECTED(failure): samba3.raw.search.sorted(nt4_dc)
   2 UNEXPECTED(error): samba3.smb2.notify.rmdir3(ad_dc) (samba.subunit.RemotedTestCase)
   2 UNEXPECTED(failure): samba4.blackbox.smbclient(s4member:local).Test login with --machine-pass without kerberos(s4member:local)
   2 UNEXPECTED(failure): samba3.raw.notify.mask(nt4_dc)
   2 UNEXPECTED(failure): samba4.drs.fsmo.python(promoted_dc).fsmo.DrsFsmoTestCase.test_SchemaMasterTransfer(promoted_dc)
   2 UNEXPECTED(failure): samba4.blackbox.dbcheck(vampire_dc).dbcheck(vampire_dc:local)
   2 UNEXPECTED(error): samba3.smb2.notify.rmdir3 (samba.subunit.RemotedTestCase)(ad_dc)
   2 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.sign (ad_member) member creds.sending a message to the remote server(ad_member)
   1 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (member) member creds.sending a message to the remote server(member)
   1 UNEXPECTED(error): samba4.drs.repl_schema.python(promoted_dc).repl_schema.DrsReplSchemaTestCase.test_classInheritance(promoted_dc)
   1 UNEXPECTED(error): samba3.smb2.durable-open.open2-lease(nt4_dc) (samba.subunit.RemotedTestCase)
   1 UNEXPECTED(failure): samba4.rpc.drsuapi on ncalrpc with validate.drsuapi.DsCrackNames(dc:local)
   1 UNEXPECTED(failure): samba4.local.event.fd1(none)
   1 UNEXPECTED(failure): samba4.tombstone_reanimation.python.tombstone_reanimation.RestoreContainerObjectTestCase.test_container(ad_dc_ntvfs)
   1 UNEXPECTED(failure): samba4.blackbox.samba_tool_demote(promoted_dc).demote(promoted_dc)
   1 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.plain (s3member) member creds.sending a message to the remote server(s3member)
   1 UNEXPECTED(failure): samba3.raw.notify.alignment(nt4_dc)
   1 UNEXPECTED(failure): samba.blackbox.wbinfo(rodc:local).wbinfo -N against rodc(rodc:local)
   1 UNEXPECTED(failure): samba3.raw.search.sorted(plugin_s4_dc)
   1 UNEXPECTED(error): samba3.smb2.durable-open.open2-lease (samba.subunit.RemotedTestCase)(nt4_dc)
   1 UNEXPECTED(error): samba4.winbind.pac.GSSAPI(ad_member:local) (samba.subunit.RemotedTestCase)
   1 UNEXPECTED(failure): samba.blackbox.wbinfo(promoted_dc:local).wbinfo -N against promoted_dc(promoted_dc:local)
   1 UNEXPECTED(error): samba4.drs.delete_object.python(promoted_dc).delete_object.DrsDeleteObjectTestCase.test_ReplicateDeletedObject2(promoted_dc)
   1 UNEXPECTED(failure): samba4.rpc.drsuapi on ncalrpc with validate.drsuapi.DsCrackNames(ad_dc_ntvfs:local)
   1 UNEXPECTED(failure): samba.blackbox.wbinfo(promoted_dc:local).wbinfo -G against promoted_dc(promoted_dc:local)
   1 UNEXPECTED(error): samba4.ldap.acl.python(fl2003dc).__main__.AclSPNTests.test_spn_rwdc(fl2003dc)
   1 UNEXPECTED(failure): samba3.raw.notify.dir(ad_dc)
   1 UNEXPECTED(failure): samba4.rpc.backupkey with seal.backupkey.unable_to_decrypt_secret(dc)
   1 UNEXPECTED(failure): samba.blackbox.wbinfo(ad_member:local).wbinfo -K against ad_member with domain creds(ad_member:local)
   1 UNEXPECTED(failure): samba3.smbtorture_s3.LOCAL-RBTREE.smbtorture(none)
   1 UNEXPECTED(failure): samba3.raw.notify.mask(ad_dc)
   1 UNEXPECTED(failure): samba4.rpc.cracknames on ncacn_ip_tcp.cracknames.cracknames-test(dc)
   1 UNEXPECTED(failure): samba3.blackbox.smbclient_s3.crypt (nt4_dc).creating a good symlink and deleting it by path(nt4_dc)
   1 UNEXPECTED(failure): samba3.smb2.create.mkdir-dup(nt4_dc)
   1 UNEXPECTED(failure): samba4.tombstone_reanimation.python.tombstone_reanimation.RestoreContainerObjectTestCase.test_container(dc)
   1 UNEXPECTED(error): samba4.drs.repl_schema.python(promoted_dc).repl_schema.DrsReplSchemaTestCase.test_all(promoted_dc)
   1 UNEXPECTED(error): samba4.winbind.pac.GSSAPI (samba.subunit.RemotedTestCase)(ad_member:local)