Improving the speed of make test

[Jelmer Vernooij]

On 03/05/2012 08:36 AM, Andrew Bartlett wrote:
> Jelmer and others interested in selftest:
>
> A while back, I started doing some profiling to determine where the time
> is spent in 'make test', using 'perf'.
>
> I was surprised to find that 15% of our time is spent in routines
> associated with SHA1, due to adding users and kinit.  Both of these run
> a *lot* of SHA1, because salting the password for the AES-based kerberos
> keys uses multiple thousands of rounds of SHA1, to make brute forcing
> the password hash harder.
>
> The fix is simple:
>   - change acl.py and similar tests not to create a user for each unit
> test, but re-use one for the whole testsuite
>   - kinit once at the start of make test, for all connections that should
> be made as administrator.  Use that credential cache for all connections
> instead of $USERNAME and $PASSWORD
>   - create another user if we ever need to modify the groups of the
> administrator (the cached PAC won't update).
>
> I've not got around to doing this yet, but as the python selftest
> rewrite is under way, I wanted to ensure this was catered for in the
> design.
Thanks.

I think one of the other issues with selftest is also that we're running 
too much high level (functional) tests rather than unit tests. We can't 
possibly run all tests with all possible permutations of Samba 
configuration options.

For example, is it useful to run all RPC tests against our own servers 
with and without the bigendian option? I can see the bigendian option 
being really useful when running tests against Windows, but our client 
and server code is generated from the same IDL - we won't find errors in 
the IDL this way. If we're trying to catch pidl bugs, I think just 
running rpc-echo with and without 'bigendian' should be sufficient, and 
more low-level tests for pidl.

Cheers,

Jelmer