CMake Proposal

[Bill Hoffman]

tridge at samba.org wrote:
> Hi Bill,
> 
> yes, I understand that cmake could call out to our current selftest
> system. Any build system that cannot call out to an external test
> program would be pretty badly broken.
> 
> Even so, the approach you've written up wouldn't work. Your example
> creates the tests as individual tests which you then loop over. You
> seem to be under the impression that the Samba test suite involves
> running invividual, standalone unit tests. We do have some tests like
> that, but the vast majority of our tests involve a much more complex
> setup.
> 
I figured I must be missing something.   :)

I had two versions, one that called the whole thing, and one that broke 
it up into smaller tests, I guess I picked the wrong one... :)

If I had gone with the version that ran the whole thing, I was not sure 
how I would pass the regular expression to the test system that lets you 
pick which tests to run.   I guess an environment variable could be used.

> When we want to test something like the SAMR functionality, we need a
> running Samba domain controller. That needs to be provisioned. We also
> need a virtual network setup that allows Samba to bind to privileged
> ports as an unprivileged user (you don't need to be root to run the
> Samba test suite). 
> 
> The test part of our current build system is actually very
> sophisticated. It is way beyond anything I have seen in any other free
> software project.
> 
It might be interesting for me to better understand how this works, and 
see if it can be generalized for other projects.  I am very interested 
in continuous integration testing.  I am also interested in making 
testing very easy for projects.
> The equivalent in KDE would be if 'make test' in KDE created a virtual
> X server and ran a full user desktop session with window managers,
> users interacting, testing of all of the desktop setting tools
> etc. All done as a non-privileged user. I'm pretty sure KDE has
> nothing like that. Neither does any other project that I am aware of
> except Samba.

Most of that does not sound like part of the testing system, but rather 
making the Samba code "testable".   Which it sounds like you have done a 
great job of doing. Basically, it sounds like you need to test a bunch 
of processes that communicate with each other.   We have done some thing 
similar with ParaView when we test it in client server and parallel 
mode.   We have to start a multi-machine (MPI) server, and test a client 
against that server.  However, we do this for each test, as the start up 
time is not that long.   It sounds like with Samba, you need to setup 
some sort of environment of processes, and then run the tests against 
them.   Is that correct?  How do you handle the issue of one test 
corrupting the environment?

Also, I am curious, are there plans to develop Samba on the Windows 
platform?

-Bill