testsuite changes

Jelmer Vernooij jelmer at samba.org
Thu Oct 19 03:32:37 GMT 2006


Hi Tridge!

On Thu, 2006-10-19 at 09:01 +1000, tridge at samba.org wrote:
> A quick question about some of the recent test suite changes.
> 
> Looking at the talloc test suite, it now runs a child process and
> execs bin/torture/LOCAL/TALLOC whereas it previously ran the test
> suite directly. This has a few consequences:
> 
>  - it means the talloc test suite is no longer run under valgrind on
>    our valgrind test host
Using valgrind on ./bin/torture/LOCAL/TALLOC or using the
--trace-children option to valgrind should fix that.

>  - you can't easily run the test under gdb

gdb can be run on the individual binary or on smbtorture with the option
'follow-fork-mode' set to 'child' (I havent tried the latter, but I
think it should work)

Alternatively, we could get smbtorture to wrap the commands it was
executing in gdb or valgrind, though I'd prefer to avoid that if
possible.

>  - it makes profiling and code coverage testing harder
Code coverage testing or profiling is doable on the individual binary (bin/torture/LOCAL/TALLOC).

>  - we get no output showing (for example) the speed of talloc vs
>    malloc, or any of the other output
This will be resolved when I commit some of my pending changes to make
smbtorture read the stderr output as well.

> I think that at least for the test suites that are written in C, I
> much prefer the old method of calling the test code directly from
> smbtorture. 
> 
> Perhaps we could have a convention of passing in a logging function
> instead?  For example:
> 
> int talloc_testsuite(void *ctx, void (*logfn)(void *ctx, const char *fmt, ...))
> {
> 	
> }
> 
> and have the old #if _SAMBA_BUILD_ with main() in it 
That would improve some things, but the advantage of using a subprocess
is:

 * identical code for the samba-build and the stand-alone build
 * being able to detect segfaults and other errors (rather than test
failures) in the testsuite
 * being able to parse the output by something else than smbtorture. For
example, we could have a script that parses the output and generates
html or graphs from it.
 * the autofree test can be run now - no sideeffects from other tests
influencing it
 * no need to relink smbtorture when changing the testsuite

The output is in a format called `subunit' that is being standardized.

Does this address your concerns? 

Cheers,

jelmer

-- 
Jelmer Vernooij <jelmer at samba.org> - http://samba.org/~jelmer/


More information about the samba-technical mailing list