[cifs-protocol] [REG:111092973873305] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

[Bryan Burgin]

Hi, Steve,

I'll look into this for you in the process of working the issue that is driving this inquiry: the large file copy to Vista or Windows 7 clients resulting in the DOS NOT_ENOUGH_MEMORY  error on subsequent SMB session setups (among other commands, like Queries, which return STATUS_INSUFF_SERVER_RESOURCES).

I have the repro of that issue and I'm presently debugging it.  I will note that on my Windows 7 system (acting as the server in this transaction), it replies with a negotiation specifying 50 (0x32) for MaxMpxCount.  Furthermore, the "Client05" system we were using, also Windows 7,  was forced to use 50 by setting Services\LanmanSERVER\Parameters\MaxMpxCt = (DWORD) 50 (previously, it returned 10 for MaxMpxCount).  It continued to fail.  (I also experimented with Services\LanmanWORKSTATION\Parameters\MaxCmds = (DWORD) 50).  Thus, the value returned by the server side via Negotiate MaxMpxCount does not seem to be linked to your initial issue.  Setting \LanmanSERVER\Parameters\MaxWorkItems, however, does seem to have a positive effect on your original issue.  And, I was able to find code that used MaxWorkItems in calculating if there was sufficient buffers for an operation.  So, there may be a direct collation yet to be verified. 

I am also aware that the failing scenario seems to be triggered by having a lot of Writes-in-flight.  In our testing together, we were seeing 15 to 20 to more Writes-in-flight, which exceeded MaxMpxCount (then 10) by quite a bit.

I'm debugging the cause of your initial issue and in the process I'll research your questions re MaxMpxCount.  Once root cause is determined, we'll know more if there was a connection between the Out-of-Memory errors and MaxMpxCount.  In the process of that work, I'll research the issues you listed below.

Bryan


-----Original Message-----
From: Christopher R. Hertel [mailto:crh at samba.org] 
Sent: Thursday, September 29, 2011 12:16 PM
To: Steve French
Cc: Interoperability Documentation Help; pfif at tridgell.net; cifs-protocol at samba.org
Subject: Re: [cifs-protocol] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

Steve,

Only Microsoft can correctly answer these questions, at this point, but I am adding my comments here to encourage them to spend a good deal of time studying both the client and server code.  The behavior of the MaxMpxCount value, in particular, is buried in the source.

My team and I spent a great deal of time digging into this particular question when writing the documentation.  We did try to make things as clear as we could.  There is an internal table, as described in [MS-CIFS], that is keyed on both PID and MID (not just MID).  This table keeps track of outstanding requests from the client.  Note that an open OpLock is an out standing request, as is a ChangeNotify.

What is difficult to determine when studying the code is how this internal table actually performs under load.

Another thing to consider is how the Windows clients handle the MaxMpxCount value.  I do not believe that they put much stress on the server, in general.

Chris -)-----

Steve French wrote:
> Resending with correct title.
> 
> On Thu, Sep 29, 2011 at 1:09 PM, Steve French <smfrench at gmail.com> wrote:
>> I would like clarification on the how the maximum number of cifs 
>> simultaneous requests are enforced, and for which cifs operations 
>> they apply to.  In part this is to avoid triggering a serious problem 
>> in Windows 7 and Windows Vista (when they are run as a server) when 
>> exceeding this limit on simultaneous requests.
>>
>> What I have observed in my testing:
>> - Although Windows 7 and Vista allows more than 10 simultaneous SMB 
>> write requests (and Windows 2008 allows more than 50 simultaneous 
>> write requests), Windows 7 and Windows Vista appear to have problems 
>> when 20 or 30 are in flight at one time.
>> - Windows does not enforce a limit on SMB Negotiate Protocol
>> - Windows seems to ignore checks on maximum simultaneous requests on 
>> certain handle based operations (SMB writeX in particular) until 
>> after the file is closed (the writes don't get an error, but the next 
>> QueryPathInfo after the file is closed gets an error).
>>
>> I need to understand on which requests (besides SMB Negotiate) I 
>> should ignore MaxMpxCount.  In particular, SMB Echo, seems like an 
>> obvious choice to ALWAYS allow to be sent by the client, since SMB 
>> Echo is sent to make sure the server is alive (when for example the 
>> limit on simultaneous requests has been reached due to slow opens, or
>> writes past end of file) and to prevent timeout.   If a client is
>> restricted from sending SMB Echo then there is no reasonably reliable 
>> mechanism available to determine whether the server is dead or just 
>> hung temporarily processing slow requests.
>>
>> MS-CIFS is not clear on which commands MaxMpxCount is enforced.
>>
>> I would like to ignore MaxMpxCount on (sending one of) SMBEcho (where 
>> you could cause data integrity issues if you limited the ability to 
>> check server state up/down) and SMBNegotiate (where it is obviously 
>> not set yet) and SMBOplockBreak responses (since you can't guarantee 
>> the order that these are received/sent from the server relative to 
>> other frames which are being processed, and can cause the server 
>> session to drop if you don't allow the client to send these) to the 
>> server since there is no reasonable mechanism to limit these without 
>> risking problems with prematurely taking down a slow session (perhaps 
>> opens of offline files for example).
>>
>> It looks like the safest way to handle this is for the client to 
>> limit pending requests to the MaxMpxCount, with the exception of the 
>> three SMB types listed above.
>>
>> In previous versions of MS-CIFS there were mentions of different 
>> processes, and uids on the negotiated tcp session and their relation 
>> to the MaxMpxCount - and I would also like to verify whether the 
>> limits are considered relative to some combination of the negotiated 
>> tcp session and uid and/or pid as earlier discussions implied (or 
>> file handle, as our testing hinted at).
>>
>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Steve
>>
> 
> 
> 

--
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/    -)-----   crh at ubiqx.org