smbtorture fails for large numbers of clients

[David Collier-Brown - Sun Canada]

After about 36-38 clients, smbtorture's NBW95 test fails.
As I was trying to do 60, this is A Bad Thing.

For people who don't use it, NBW95 is a simulation of the netbench
test with Windows 95 clients.  It exists so you don't have to borrow
a lab with 60 PCs to do simple load tests.

The symptom is this:
many lines of "(1588) write failed on handle 4253"
followed by "(1651) nb_read: handle 4255 was not open 
size=4096 ofs=0" 



Ok, time to truss the failing tests. 
---
PID	Command					Return code
===	=======					===========
8622:   send(3, "\0\002A7FF S M B\v\0\0\0".., 683, 0)   = 683
8552:   recv(3, "\0\0\0 #", 4, 0)               = 4
8565:   recv(3, "\0\0\0 7", 4, 0)               = 4
8622:   poll(0xFFBEE488, 1, 120000)             = 1
8645:   recv(3, "FF S M B\b\0\0\0\08801\0".., 55, 0)    = 55
8552:   recv(3, "FF S M B\v01\006\08801\0".., 35, 0)    = 35
8565:   recv(3, "FF S M B\b\0\0\0\08801\0".., 55, 0)    = 55
8622:   recv(3, "\0\0\0 %", 4, 0)               = 4
8645:   send(3, "\0\0\0 )FF S M B04\0\0\0".., 45, 0)    = 45
8618:   poll(0xFFBEE488, 1, 120000)             = 1
8645:   poll(0xFFBEE488, 1, 120000)             = 1
8622:   recv(3, "FF S M B\v\0\0\0\08801\0".., 37, 0)    = 37
8552:   write(1, " ( 1 5 8 8 )   w r i t e".., 35)      = 35

That sequence ended in "(1588) write failed on handle 4253". 

This was followed by 20 more problem reports on 4534, and then by 
"(1651) nb_read: handle 4255 was not open size=4096 ofs=0":

8572:   recv(3, "\0\0\0 %", 4, 0)               = 4
8552:   recv(3, "\0\0\0 #", 4, 0)               = 4
8572:   recv(3, "FF S M B\v\0\0\0\08801\0".., 37, 0)    = 37
8552:   recv(3, "FF S M B0701\005\08801\0".., 35, 0)    = 35
8622:   send(3, "\0\010 0FF S M B\v\0\0\0".., 4148, 0)  = 4148
8622:   poll(0xFFBEE488, 1, 120000)             = 1
8552:   write(1, " ( 1 6 5 1 )   n b _ r e".., 57)      = 57

A quick look at the code in smbtorture, dbench and tbench exposed a 
define MAX_FILES, which was being exceeded and which produced
the "nb_read: handle 4255 was not open" message.  Alas, I increased
it by 100 and got to run 4 more clients worth of load (:-()

4 clients started
...(107) write failed on handle 4187
................................................................................
.....(5950) write failed on handle 4287
(5952) write failed on handle 4287
(5954) write failed on handle 4287
(5956) write failed on handle 4287
(5958) write failed on handle 4287
(5960) write failed on handle 4287
(5962) write failed on handle 4287
(5964) write failed on handle 4287
(5966) write failed on handle 4287
(5968) write failed on handle 4287
(5970) write failed on handle 4287
(5972) write failed on handle 4287
(5974) write failed on handle 4287
(5976) write failed on handle 4287
(5978) write failed on handle 4287

Now, the code looks sane: open adds an fd to the table controlled by
MAX_FILES, and all the other operations  search it.

Anyone else used smbtorture/dbench/tbench? 
I think they're cool, as I can replay an arbitrary load captured
from real clients, and want to use them in a sizing exercise.

--dave
--
David Collier-Brown in Boston
Phone: (781) 442-0734, Room BUR03-3632