Test infrastructure improvements

[Jelmer Vernooij]

Matthieu and I got together a couple of weeks ago to hack on the
Samba testsuite.

Our main goal has so far been on reducing the overall run time of the
testsuite, through various means. Matthieu has been pesking me about
running tests in parallel for a long time.

I've put the todo-list we compiled on the wiki:
https://wiki.samba.org/index.php/Test_Infrastructure_Sprint

What we have at the moment:

* drop the samba3/samba4 distinction?
* randomize test order [mat]
* statistics!
 * measure total run time and per test run time
 * prevent slow test addition
 * record env startup time [jelmer]
* make testsuite output top X [mat]
* reduce the cardinality of RPC tests
* parallelize with testr [jelmer]
 * migrate to subunit2
 * make all tests support --list
 * <s>require all tests output subunit</s>
 * reduce selftest run overhead
  * <s>make symbol deduplication a test, not a build phase</s>
* documentation
* simplification of test run infrastructure

Cheers,

Jelmer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141025/512ca909/attachment.pgp>