[PATCH] Selftest improvements

[Jamie McClymont]

On 07/02/18 11:01, Jamie McClymont via samba-technical wrote:
> At your request, I'm running some private autobuilds now, attempting to
> confirm that there are no differences in which tests are run.


As some validation that these patches do not reduce the number of tests
being run, below are the sizes of the stdout files for each autobuild
job with none, the first, and both of my commits.

The only one with a decrease on the first patch is systemkrb5: this is
due to the change in the treatment of samba3 (non-samba3 tests are no
longer printed when running a test without an ad_dc).

Two jobs drop in size with both patches: these are due to the skipped
environments, for which test names are also no longer printed.


Additionally, at the bottom are the runtimes for the selftest.pl-based
jobs: these do not change to a statistically significant degree,
implying that these patches had no practical effect on the tests being
run, as intended.


B = a second identical run, which I forgot to do for the 2patches version.


File sizes:
ctdb              0patches   930070
ctdb              0patchesB  929232
ctdb              1patch     928900
ctdb              1patchB    925582
ctdb              2patches   925438
ldb               0patches   52067
ldb               0patchesB  52067
ldb               1patch     52067
ldb               1patchB    52067
ldb               2patches   52067
pidl              0patches   7522
pidl              0patchesB  7522
pidl              1patch     7522
pidl              1patchB    7522
pidl              2patches   7522
replace           0patches   447
replace           0patchesB  447
replace           1patch     447
replace           1patchB    447
replace           2patches   447
samba             0patches   202337
samba             0patchesB  202244
samba             1patch     202502
samba             1patchB    202239
samba             2patches   202372
samba-ctdb        0patches   1222
samba-ctdb        0patchesB  1222
samba-ctdb        1patch     1222
samba-ctdb        1patchB    1222
samba-ctdb        2patches   1222
samba-libs        0patches   30694
samba-libs        0patchesB  30694
samba-libs        1patch     30694
samba-libs        1patchB    30694
samba-libs        2patches   30692
samba-nopython    0patches   1005
samba-nopython    0patchesB  1005
samba-nopython    1patch     1005
samba-nopython    1patchB    1005
samba-nopython    2patches   1005
samba-o3          0patches   42895
samba-o3          0patchesB  43007
samba-o3          1patch     42922
samba-o3          1patchB    42948
samba-o3          2patches   16454
samba-static      0patches   2596
samba-static      0patchesB  2596
samba-static      1patch     2596
samba-static      1patchB    2596
samba-static      2patches   2596
samba-systemkrb5  0patches   183950
samba-systemkrb5  0patchesB  183964
samba-systemkrb5  1patch     52840
samba-systemkrb5  1patchB    53090
samba-systemkrb5  2patches   11046
samba-xc          0patches   0
samba-xc          0patchesB  0
samba-xc          1patch     0
samba-xc          1patchB    0
samba-xc          2patches   0
talloc            0patches   27845
talloc            0patchesB  27875
talloc            1patch     27877
talloc            1patchB    27847
talloc            2patches   27627
tdb               0patches   1860
tdb               0patchesB  1860
tdb               1patch     1860
tdb               1patchB    1860
tdb               2patches   1858
tevent            0patches   649
tevent            0patchesB  649
tevent            1patch     649
tevent            1patchB    649
tevent            2patches   649


Times (selftest.pl-based):
samba             0patches   5h4m3s
samba             0patchesB  5h18m7s
samba             1patch     5h15m22s
samba             1patchB    5h7m46s
samba             2patches   5h15m36s
samba-o3          0patches   4m51s
samba-o3          0patchesB  5m8s
samba-o3          1patch     5m28s
samba-o3          1patchB    5m15s
samba-o3          2patches   5m21s
samba-systemkrb5  0patches   53s
samba-systemkrb5  0patchesB  1m0s
samba-systemkrb5  1patch     43s
samba-systemkrb5  1patchB    1m2s
samba-systemkrb5  2patches   46s