[scottprive@earthlink.net: Re: Information on Samba QA process, Regression testsuites]

[Martin Pool]

Please let's have this conversation on the list.

> I have Martin's "notes.txt" (Notes on Samba Testing Framework for
> Unittests). I think I misunderstood, and assumed there was a  working (if
> immature) basic framework that hadn't been checked in yet.
> 
> OK, we're at the planning stage? Some questions:

We have some tiny tests but are basically still in planning.

Tim and myself, and other people, are getting very annoyed with the
lack of tests and we want to fix it soon.

I'm going to commit our code into HEAD today or tomorrow.

> 1) For a supporting test framework, have you looked at STAF? I wish I had
> looked into STAF when I started my system tests. I ended up writing a whole
> lot of ssh/expect-driven functions to remotely reconfigure servers and
> clients, and of course installing/configuring Cygwin ssh on each NT
> box.

This is the IBM suite?

I think I've seen it before.  I'll have a look at it. 

I feel a bit of "NIH" and would like a simple framework customized to
our needs, but I'll try to look at STAF with an open mind.

> The above example is generating actions on clients, but STAF can also be
> used as an interface to maintain a resource pool. This was the first problem
> you were trying to solve, correct?
> 
> 2) What value do you see in a distributed system suite, one that uses the
> same code path as clients... ex: a NT client using "net use" to attach a
> remote drivemap?

Yes, I think both approaches are valuable.

Driving NT clients is perhaps a bit harder: there needs to be some
means to coordinate and drive them remotely.

Testing via Samba client gives a somewhat less valid test, because
there could be corresponding bugs in both our client and server.  But
it might be easier.

> I am wondering if system tests are redundant, but it's something I can code
> alone since I understand the requirements. For lower-level tests, I
> understand less and would wait for your first tests so I could study it, and
> then I'd need a suggested test to automate.
> 
> 3) In the meantime of unit-planning, what else can do to help out? I already
> have a network environment I can work in here: one 2K PDC, an XP client, RH
> 8.0, plus on instance of VMWare on each box if I need older versions of
> Windows, etc.

Learn Python :-)

Write a draft test plan for some aspect of Samba.

Point out problems that we'll hit in the future that we haven't
thought of yet.

...?

-- 
Martin