smbtorture - why has it's output format changed?

Jelmer Vernooij jelmer at
Fri Nov 24 12:37:40 GMT 2006

On Fri, 2006-11-24 at 13:17 +1100, Gary R. Schmidt wrote:
> Quoting Jelmer Vernooij <jelmer at>:
> > If you are looking for machine-parseable output that is guaranteed
> > to
> > stay the same, try the --format=subunit option.
> Hmmm, it wraps the sub-tests, but I want to call each test by name.
Can you give an example?

> > > I can understand adding tests, and I can almost understand changing
> > it 
> > > to allow sub-tests, but removing tests?  No, you don't do that.
> > I don't think we've actually removed any tests. Which ones are you
> > missing?
> Missing tests:
> I understand that some of these may have been subsumed by other tests
> (BASE-LOCK? appear to now be part of BASE-LOCK), but why should I (or
> *anyone*) have to wade through the parameter list looking for possible
> replacements?
BASE-LOCK* has indeed been integrated into BASE-LOCK. Some of the other
tests have been renamed.

Is there any reason you can't use Samba3's "make test"? That should run
all tests we know are working against Samba 3 and look at the exit code
to see if anything broke.

> > Also, Samba 4 is still under heavy development. Things like this may
> > and
> > will change over time. We don't make any guarantees.
> Changes to the output of a test tool are to be expected as new tests are
> added, but a wholesale change in format is just wrong.

> Back in the dim and dark when I was taught how to do software, test
> tools were designed first, and changes to output *format* was only
> accepted after long and intense debate, as we also designed tools that
> took that output and told us many useful things, such as whether the
> total number of failures was increasing or decreasing.
I disagree. smbtorture's default output is intended to be readable by
and useful to humans; there was no intention to have the output be
machine-parseable at all. All we rely on is the exit code. 


Jelmer Vernooij <jelmer at> -
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :

More information about the samba-technical mailing list