scan tests in make test (was Re: Retiring or fixing smbtorture?)

[Andrew Bartlett]

On Tue, 2012-10-23 at 15:43 -0700, Richard Sharpe wrote:
> Hi Folks,
> 
> Shouldn't we either retire smbtorture or fix it?
> 
> For example, the TRANS2SCAN test fails when a call is made to
> cli_open/cli_openx (depending on version) to open a directory ("\\")
> but the underlying code sets the file attributes to not a directory.
> 
> This is just bit rot. Should it be fixed? It could probably be simply
> fixed by adding a file_attributes parameter to the calls made all the
> way down.

So, some of the history here is that we don't run scan tests in make
test, because they really need to be run against windows.  We then don't
run them against windows because we now have an official way of
discovering new protocol elements, rather than scanning and guessing.

Part of the reason we don't run them in make test is that they tend to
be fairly slow, but because they don't validate their output they are
not particularly useful, except in filling logs with unexpected command
messages.

The blocking of these is done by the selftest/skip file, and comments on
some other scan entries include:

^samba4.rpc.autoidl  # this one just generates a lot of noise, and is no
longer useful
^samba4.rpc.countcalls # this is not useful now we have full IDL
^samba4.rap.scan # same thing here - we have docs now
^samba4..*trans2.scan # uses huge number of file descriptors
^samba4.*.base.scan.ioctl # bad idea in make test
^samba4.*.base.scan.pipe_number # bad idea in make test

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org