testsuite changes
Jelmer Vernooij
jelmer at samba.org
Thu Oct 19 03:32:37 GMT 2006
Hi Tridge!
On Thu, 2006-10-19 at 09:01 +1000, tridge at samba.org wrote:
> A quick question about some of the recent test suite changes.
>
> Looking at the talloc test suite, it now runs a child process and
> execs bin/torture/LOCAL/TALLOC whereas it previously ran the test
> suite directly. This has a few consequences:
>
> - it means the talloc test suite is no longer run under valgrind on
> our valgrind test host
Using valgrind on ./bin/torture/LOCAL/TALLOC or using the
--trace-children option to valgrind should fix that.
> - you can't easily run the test under gdb
gdb can be run on the individual binary or on smbtorture with the option
'follow-fork-mode' set to 'child' (I havent tried the latter, but I
think it should work)
Alternatively, we could get smbtorture to wrap the commands it was
executing in gdb or valgrind, though I'd prefer to avoid that if
possible.
> - it makes profiling and code coverage testing harder
Code coverage testing or profiling is doable on the individual binary (bin/torture/LOCAL/TALLOC).
> - we get no output showing (for example) the speed of talloc vs
> malloc, or any of the other output
This will be resolved when I commit some of my pending changes to make
smbtorture read the stderr output as well.
> I think that at least for the test suites that are written in C, I
> much prefer the old method of calling the test code directly from
> smbtorture.
>
> Perhaps we could have a convention of passing in a logging function
> instead? For example:
>
> int talloc_testsuite(void *ctx, void (*logfn)(void *ctx, const char *fmt, ...))
> {
>
> }
>
> and have the old #if _SAMBA_BUILD_ with main() in it
That would improve some things, but the advantage of using a subprocess
is:
* identical code for the samba-build and the stand-alone build
* being able to detect segfaults and other errors (rather than test
failures) in the testsuite
* being able to parse the output by something else than smbtorture. For
example, we could have a script that parses the output and generates
html or graphs from it.
* the autofree test can be run now - no sideeffects from other tests
influencing it
* no need to relink smbtorture when changing the testsuite
The output is in a format called `subunit' that is being standardized.
Does this address your concerns?
Cheers,
jelmer
--
Jelmer Vernooij <jelmer at samba.org> - http://samba.org/~jelmer/
More information about the samba-technical
mailing list