About faster make test

[Matthieu Patou]

On 05/15/2014 04:55 AM, Jelmer Vernooij wrote:
> On Wed, May 14, 2014 at 03:11:48PM -0700, Matthieu Patou wrote:
>> I'm quoting another email:
>> "Proper unit tests. The current make test setup is great, but that's
>> basically all integration tests. I'd like to discuss adding a proper unit
>> test suite that then hopefully can be run faster than the current tests. "
>>
>> I'm not opposing to unit tests, but we could also make test run faster today
>> by resolving this bug:https://bugzilla.samba.org/show_bug.cgi?id=8579
>>
>> My laptop nowdays  have 4 cores, and having 8 cores in a workstation costs
>> less than $700, so running environments in parallel should really speed up
>> the whole test.
> We've had this discussion a couple of times before over the last couple of
> years. :-) Parallelising 'make test' will help the runtime, but at this point
> distracts us from improving our overall test strategy - as Kai says, we
> emphasize integration tests too much.
>
> Adding parallellisation can probably get us a 2x-3x win on quad-core laptops
> (since we already have more than one process active per test run, e.g. one server
> and one smbtorture instance). With the current rate that we're adding integration
> tests, we'll probably be back where we started (testsuite runtime-wise) in a
> year or so.
Well I'm not opposing this to the move to smaller tests, I was more 
proposing that first we make what we have today faster and maybe we 
commit to do more real unit tests to reduce to keep our testing time 
under control.
Please also keep in mind that on sn where we have lots of CPU this will 
lead to much faster test time, nowdays server give you 64 threads (32 
cores) so it might be a solution to have our make tests running on 
sn-like machine.

IHMO it's a 3 items thing:

* make test run in parallel
* identify tests that are slow and why (maybe reduce the number of time 
we test all the combination of big/little endian sign/unsigned, ...)
* have real low level unit test framework so that you can choose between 
end to end tests and real unit tests

Matthieu
> Cheers,
>
> Jelmer

-- 
Matthieu Patou
Samba Team
http://samba.org