smbtorture - why has it's output format changed?

[Jelmer Vernooij]

On Fri, 2006-11-24 at 13:17 +1100, Gary R. Schmidt wrote:
> Quoting Jelmer Vernooij <jelmer at samba.org>:
> > If you are looking for machine-parseable output that is guaranteed
> > to
> > stay the same, try the --format=subunit option.
> Hmmm, it wraps the sub-tests, but I want to call each test by name.
Can you give an example?

> > > I can understand adding tests, and I can almost understand changing
> > it 
> > > to allow sub-tests, but removing tests?  No, you don't do that.
> > I don't think we've actually removed any tests. Which ones are you
> > missing?
> Missing tests:
[...]
> I understand that some of these may have been subsumed by other tests
> (BASE-LOCK? appear to now be part of BASE-LOCK), but why should I (or
> *anyone*) have to wade through the parameter list looking for possible
> replacements?
BASE-LOCK* has indeed been integrated into BASE-LOCK. Some of the other
tests have been renamed.

Is there any reason you can't use Samba3's "make test"? That should run
all tests we know are working against Samba 3 and look at the exit code
to see if anything broke.

> > Also, Samba 4 is still under heavy development. Things like this may
> > and
> > will change over time. We don't make any guarantees.
> Changes to the output of a test tool are to be expected as new tests are
> added, but a wholesale change in format is just wrong.

> Back in the dim and dark when I was taught how to do software, test
> tools were designed first, and changes to output *format* was only
> accepted after long and intense debate, as we also designed tools that
> took that output and told us many useful things, such as whether the
> total number of failures was increasing or decreasing.
I disagree. smbtorture's default output is intended to be readable by
and useful to humans; there was no intention to have the output be
machine-parseable at all. All we rely on is the exit code. 

Cheers,

Jelmer
-- 
Jelmer Vernooij <jelmer at samba.org> - http://samba.org/~jelmer/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20061124/addcd0e2/attachment.bin