Improving the speed of make test
jelmer at samba.org
Mon Mar 5 16:33:26 MST 2012
Am 05/03/12 23:50, schrieb Matthieu Patou:
> On 03/05/2012 12:23 AM, Volker Lendecke wrote:
>> On Mon, Mar 05, 2012 at 06:36:57PM +1100, Andrew Bartlett wrote:
>>> Jelmer and others interested in selftest:
>>> A while back, I started doing some profiling to determine where the
>>> is spent in 'make test', using 'perf'.
>>> I was surprised to find that 15% of our time is spent in routines
>> Is that total time or CPU time? Quite some of the tests in
>> the file server area look at or depend on timeout behaviour.
>> The tests for share modes for example come to mind. With
>> some care (independent file names), they could probably very
>> well run in parallel and gain quite a bit of total time.
> I suggested here :https://bugzilla.samba.org/show_bug.cgi?id=8582
> that we run tests in environment in parallel (ie. samba3.* tests while
> doing samba4.* dc and samba4 fl2000, ...) on machines with more that
> 2 cores this could prove to be very valuable I'm sure.
We actually have the infrastructure for this - you can do most of it
today using testrepository or with the standard subunit tools and some
Basically, you can list a specific set of tests to run in a file and
then run those tests by using the --load-list option to "waf selftest".
samba.tests.xattr.TestCopyTreeWithXattrs.test_simple > todo
samba.tests.xattr.XattrTests.test_set_and_get_native >> todo
gwenhwyvar:~/src/samba% ./buildtools/bin/waf test --load-list=todo
The --list option can be used to get a list of all available tests:
gwenhwyvar:~/src/samba% ./buildtools/bin/waf test --list
You can get a list of all failed tests in the previous test run with
gwenhwyvar:~/src/samba% subunit-filter --no-passthrough --no-skip
--no-success < st/subunit | subunit-ls > failed-tests
The result of that command could be used to run all failed tests:
gwenhwyvar:~/src/samba% ./buildtools/bin/waf test --load-list=failed-tests
The main thing that has to be done to be able to use this to be able to
run subsets of the testsuite (such as all previously failing tests) is
converting the remaining tests that don't understand the --list and/or
--load-list options. selftest will warn about those if you run it with
The other thing that's blocking parallel running of the testsuite is the
fact that environments are in no way shareable at the moment. Thatwill
hopefully change with my selftest.py work.
More information about the samba-technical