SOC Automated Windows Testing Project

tridge at samba.org tridge at samba.org
Wed May 31 04:20:47 GMT 2006


Brad,

 > Hi Tridge, i've been looking at starting the listening service on the 
 > windows side that sets up and tears down the windows configurations for 
 > each test. I've decided to go with python and a simple bsd socket 
 > listener for now sending ascii text strings. Do you have any thoughts 
 > about this approach?

I think the main decision you need to make is whether you will 'drive'
the testing from the unix side (the side running 'make test' in Samba)
or from the windows side.

The little python listener you describe is an example of driving the
testing from the windows side, as most of the logic of the tests would
be written using scripts that would execute on windows.

The other approach would be to assume the windows box has some sort of
remote terminal capability (eg. telnet server, ssh server or similar),
and then drive that via an expect script (or similar) from the unix
side.

This has the advantage that all the testing would be driven by the
machine running the 'make test'. So somewhere in our tests scripts
we'd have something like:

 SETUP_TESTING_LINK
 SEND "net use z: \\server\share"
 EXPECT "The command completed successfully"

(I'm just making up some pseudo-code syntax above - real expect
scripting would look a bit different)

I think both approaches have their advantages. The main advantage I
see of the expect approach is that it might require less stuff to be
initially installed/maintained on the windows box (so no win32-python
for example). The advantage of the approach you've suggested is it
might be easier to do better error checking.

It might be an idea to run with what you've started on for a few days,
see how it works out, and then once you have tried it a bit see if you
think it will be workable, or if you want to change approach to a more
expect/response based system.

 > For the RAW-QFILEINFO smbtorture test, i've written .vbs scripts to add 
 > and delete a user, a script which creates a directory and share it to a 
 > user, and one to remove the share and the directory. The scripts need to 
 > do alot more return code checking and error reporting.

Interesting! I haven't seen python used to manipulate ADS objects like
this before.

(for those on samba-technical but without a checkout of the SOC svn
tree, see http://samba.org/ftp/unpacked/SOC/bnh/ for a copy of these
scripts)

Why did you choose to use ADS calls rather than calling to the command
line tools? I was imagining you'd do things like:

  net user USERNAME PASSWORD /ADD
  net user USERNAME /DELETE
  net share test=c:\test /remark:"test share"

Did you choose the ADS calls to get better error checking? That would
certainly be a valid reason, but it would also make it harder to run
these tests on Win9X, WinNT4 and other OSes that don't have ADS.

Or maybe you need more control over the properties of the users/shares
than you can get with net.exe commands?

 > 1. Samba test connects to the listening windows host and passes an ascii 
 > string containing the name of the test and relevant parameters.
 > 2. Windows host creates a user account and a share as requested. That 
 > user is configured on the windows host to have full control of that share.
 > 3. Windows host replys over the socket with an error code to indicate 
 > success or failure.
 > 4. Samba test runs "smbtorture -W TESTDOMAIN -U testuser%password 
 > //dc0.test.domain/testshare RAW-QFILEINFO"
 > 5. Samba test replies to windows host and windows host removes user, 
 > share and directory.
 > 6. Windows host replies success or failure of (5).

yep, the trick there will be to recover from situations where the user
already exists, or the share exists, or the password is rejected
etc. You could either cope with this by assuming the test always
starts off with a clean setup (due to a vmware snapshot restore for
example) or you could catch the errors and fix them (eg. delete old
user if user already exists etc).


 > 7. Samba test creates a directory, and a user account, and a smb.conf 
 > which shares that directory, giving that user full control.
 > 8. Samba test replies to Windows host with the test name, username, 
 > password and share name.
 > 9. Windows host runs equivalent RAW-QFILEINFO test vs smbd and replies 
 > with its results to Samba test.
 > 10. Disconnect. Samba test halts smbd and removes directory and smb.conf 
 > file.

for this half of the project (the windows->samba half) I think we will
be mostly testing existing windows commands, like net, xcopy, del,
cacls etc, rather than having anything close to RAW-QFILEINFO. We
could port smbtorture to windows (its been done before), but I don't
think it would win us much, as it would use our SMB library code, and
what we really want to test is the Windows native SMB library code
against a Samba server.

We could try for something like ifstest, and certainly running
automated ifstest would be fantastic, but lets try for the simpler
stuff first (if you haven't heard of ifstest let me know and I can
give you some background on that).

Alternatively, you could use interfaces between things like python and
win32. It would be important not to use things like cygwin, as those
tend to make quite different types of calls to what native win32 apps
use. 

Cheers, Tridge


More information about the samba-technical mailing list