[clug] Tests

[David Howe]

This might be helpful...

http://swik.net/smbtorture


____
David Howe
____________
http://www.qednet.biz
_____________________





On 23/05/2008, at 7:57 AM, Brad Hards wrote:

> On Thursday 22 May 2008 10:43:46 pm Sjors Provoost wrote:
>> Tridge showed a brilliant piece of testing software after pizza time.
>> I would definitely like to try something like that on the software I
>> need to write the coming months. What is it and where can I find it?
>
> It is "gentest", which is part of the samba torture suite.
> http://git.samba.org/?p=samba.git;a=tree;f=source/torture;hb=v4-0-test
>
> (Note: everything below this is potentially wrong - I had to work  
> pretty hard
> just to keep up with what Tridge was telling us).
>
> Tridge was showing "gentest_smb2" which is the version of the tool  
> for the
> SMB2 protocol (or at least the file server parts that samba4 will  
> implement -
> not sure it is meant to cover the whole protocol). There is a  
> version for the
> original smb protocol too (just called "gentest"), and a special  
> version for
> oplock testing (called locktest). All of those are in the samba4  
> torture/
> directory - see gitweb link above.
>
> As I understand it, the test does semi-intelligent comparisons  
> between two
> servers (say A and B). The oplock version opens two connections to  
> each
> server so you can check locking results - not sure if the
> gentest/gentest_smb2 versions do that yet.
>
> The idea is that if you do the same operation on each server (A and  
> B) - say a
> CreateFile() type operation, and the results differ, then something  
> is wrong.
> Of course, that type of thing is pretty easy to find using some  
> basic tests.
> Now envisage a sequence of random-ish operations (A0...An, and  
> B0...Bn).
> Perform those on both servers, and you might get a difference at  
> step n.
> Since that could be really large, it doesn't make a very good test.  
> However
> of those n operations, some subset are probably irrelevant. What  
> gentest does
> is a bisection search (what did you expect from the guy with a Ph.D in
> searching and sorting algorithms) that identifies a smaller set of  
> operations
> that actually replicate the problem. [I'm not sure it provably is
> the "smallest set", because there theoretically could be multiple  
> paths, but
> that probably doesn't occur too much in reality.] There is a chance  
> that the
> server behaviour is not deterministic (e.g. if you replay the same  
> sequence
> of operations, it doesn't break in the same place, aka a bug), and  
> that will
> break the search.
>
> The other part that makes it very smart is that it encapsulates a  
> lot of
> Tridge level knowledge about how the protocol works, and what is  
> needed to
> explore the API space.  For example, when you ask Server A and  
> Server B for
> an operation that gives you back a file handle, you should expect  
> that they
> might be different (and that isn't an error) and that you'll need  
> to pass the
> right handle back to the right server for subsequent operations.  
> Same for
> results on files that pass back creation time (should be close, but  
> won't be
> identical).
>
> The idea that you have two servers and compare results is pretty  
> fundamental
> to gentest and friends. I'm not sure how well that will apply to your
> situation, but I wish you good luck!
>
> Brad
> -- 
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux