[cifs-protocol] [REG:111092973873305] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

Wed Oct 26 10:32:12 MDT 2011

Hi Steve

Per our analysis, MaxMpxCount is enforced on all the commands including Negotiate, Echo and SMBOplockBreak. Please find the answers as follows :

1.     I would like clarification on the how the maximum number of cifs  simultaneous requests are enforced, and for which cifs operations  they apply to.  In part this is to avoid triggering a serious problem  in Windows 7 and Windows Vista (when they are run as a server) when  exceeding this limit on simultaneous requests.

                On both the client and the server, MaxMpxCount is used to limit the number of outstanding commands Per SMB Connection.  This behavior applies to ALL commands.

2.    What I have observed in my testing:
        - Although Windows 7 and Vista allows more than 10 simultaneous SMB  write requests (and Windows 2008 allows more than 50 simultaneous  write requests), Windows 7 and Windows Vista appear to have problems  when 20 or 30 are in flight at one time.

We would be requiring capture for this scenario for further analysis.

3.  Windows does not enforce a limit on SMB Negotiate Protocol
      Windows seems to ignore checks on maximum simultaneous requests on  certain handle based operations (SMB writeX in particular) until  after the file is closed (the writes don't get an error, but the next  QueryPathInfo after the file is closed gets an error).

              Windows in fact sends STATUS_DATA_NOT_ACCEPTED. Please let us know if you are not getting this error.

4. I need to understand on which requests (besides SMB Negotiate) I  should ignore MaxMpxCount.  In particular, SMB Echo, seems like an  obvious choice to ALWAYS allow to be sent by the client, since SMB
    Echo is sent to make sure the server is alive (when for example the  limit on simultaneous requests has been reached due to slow opens, or writes past end of file) and to prevent timeout.   If a client is restricted from sending SMB Echo then there is no reasonably reliable  mechanism available to determine whether the server
    is dead or just  hung temporarily processing slow requests.

            To really have a problem with SMB Echo or any other commands, total pending requests(including SMB Echo) should be greater than what server could really handle, which might be possible only when server is really busy. The above mentioned scenario would not arise unless server is busy.
    In this scenario, SMB should fail Echo if it cannot process more requests. Client can keep retrying to check if server is available.

5. MS-CIFS is not clear on which commands MaxMpxCount is enforced.

             MaxMpxCount will be enforced for all the commands in a particular Connection.

6. I would like to ignore MaxMpxCount on (sending one of) SMBEcho (where  you could cause data integrity issues if you limited the ability to check server state up/down) and SMBNegotiate (where it is obviously
    not set yet) and SMBOplockBreak responses (since you can't guarantee  the order that these are received/sent from the server relative to  other frames which are being processed, and can cause the server
    session to drop if you don't allow the client to send these) to the  server since there is no reasonable mechanism to limit these without  risking problems with prematurely taking down a slow session (perhaps
    opens of offline files for example).

            SMBEcho, SMB Negotiate & SMB Oplock response are NOT excluded from MaxMpxCount limit. However the server does not enable Oplocks if MaxMpxCount = 1, as Oplock break notification/ack need an extra work item when current create/lock/write request is pending on oplock break.

7. It looks like the safest way to handle this is for the client to  limit pending requests to the MaxMpxCount, with the exception of the  three SMB types listed above.

            Client would always limit the pending requests to MaxMpxCount, without the exception of any commands.

8. In previous versions of MS-CIFS there were mentions of different  processes, and uids on the negotiated tcp session and their relation  to the MaxMpxCount - and I would also like to verify whether the  limits are considered relative to some combination of the negotiated  tcp session and uid and/or pid as earlier discussions
    implied (or  file handle, as our testing hinted at).

            MaxMpxCount on the client is based on the outstanding PID/MID pairs in the Client.Connection.PIDMIDList per SMB Connection.

Please let us know if any further clarification is required.

Thanks

Tarun Chopra.

-----Original Message-----
From: Bryan Burgin
Sent: Friday, October 14, 2011 1:20 PM
To: 'Steve French'; Tarun Chopra
Cc: MSSolve Case Email; 'pfif at tridgell.net'; 'cifs-protocol at samba.org'
Subject: RE: [REG:111092973873305] [cifs-protocol] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

[Adding Tarun]

Steve,

I am working with Windows Sustained Engineering on the Windows Vista and Windows 7 issue you reported.  It is not directly related to MaxMpx, which is the topic of this thread.

Tarun is researching your questions re "I need to understand on which requests (besides SMB Negotiate) I should ignore MaxMpxCount", "MS-CIFS is not clear on which commands MaxMpxCount is enforced" and other protocol-related MaxMpx questions.

Bryan

-----Original Message-----

From: Bryan Burgin

Sent: Thursday, September 29, 2011 2:19 PM

To: Steve French

Cc: MSSolve Case Email; pfif at tridgell.net<mailto:pfif at tridgell.net>; cifs-protocol at samba.org<mailto:cifs-protocol at samba.org>

Subject: [REG:111092973873305] [cifs-protocol] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

Hi, Steve,

I'll look into this for you in the process of working the issue that is driving this inquiry: the large file copy to Vista or Windows 7 clients resulting in the DOS NOT_ENOUGH_MEMORY  error on subsequent SMB session setups (among other commands, like Queries, which return STATUS_INSUFF_SERVER_RESOURCES).

I have the repro of that issue and I'm presently debugging it.  I will note that on my Windows 7 system (acting as the server in this transaction), it replies with a negotiation specifying 50 (0x32) for MaxMpxCount.  Furthermore, the "Client05" system we were using, also Windows 7,  was forced to use 50 by setting Services\LanmanSERVER\Parameters\MaxMpxCt = (DWORD) 50 (previously, it returned 10 for MaxMpxCount).  It continued to fail.  (I also experimented with Services\LanmanWORKSTATION\Parameters\MaxCmds = (DWORD) 50).  Thus, the value returned by the server side via Negotiate MaxMpxCount does not seem to be linked to your initial issue.  Setting \LanmanSERVER\Parameters\MaxWorkItems, however, does seem to have a positive effect on your original issue.  And, I was able to find code that used MaxWorkItems in calculating if there was sufficient buffers for an operation.  So, there may be a direct collation yet to be verified.

I am also aware that the failing scenario seems to be triggered by having a lot of Writes-in-flight.  In our testing together, we were seeing 15 to 20 to more Writes-in-flight, which exceeded MaxMpxCount (then 10) by quite a bit.

I'm debugging the cause of your initial issue and in the process I'll research your questions re MaxMpxCount.  Once root cause is determined, we'll know more if there was a connection between the Out-of-Memory errors and MaxMpxCount.  In the process of that work, I'll research the issues you listed below.

Bryan

-----Original Message-----

From: Christopher R. Hertel [mailto:crh at samba.org]<mailto:[mailto:crh at samba.org]>

Sent: Thursday, September 29, 2011 12:16 PM

To: Steve French

Cc: Interoperability Documentation Help; pfif at tridgell.net<mailto:pfif at tridgell.net>; cifs-protocol at samba.org<mailto:cifs-protocol at samba.org>

Subject: Re: [cifs-protocol] [Pfif] Maximum Simultaneous Request for CIFS behavior clarification

Steve,

Only Microsoft can correctly answer these questions, at this point, but I am adding my comments here to encourage them to spend a good deal of time studying both the client and server code.  The behavior of the MaxMpxCount value, in particular, is buried in the source.

My team and I spent a great deal of time digging into this particular question when writing the documentation.  We did try to make things as clear as we could.  There is an internal table, as described in [MS-CIFS], that is keyed on both PID and MID (not just MID).  This table keeps track of outstanding requests from the client.  Note that an open OpLock is an out standing request, as is a ChangeNotify.

What is difficult to determine when studying the code is how this internal table actually performs under load.

Another thing to consider is how the Windows clients handle the MaxMpxCount value.  I do not believe that they put much stress on the server, in general.

Chris -)-----

Steve French wrote:

> Resending with correct title.

>

> On Thu, Sep 29, 2011 at 1:09 PM, Steve French <smfrench at gmail.com<mailto:smfrench at gmail.com>> wrote:

>> I would like clarification on the how the maximum number of cifs

>> simultaneous requests are enforced, and for which cifs operations

>> they apply to.  In part this is to avoid triggering a serious problem

>> in Windows 7 and Windows Vista (when they are run as a server) when

>> exceeding this limit on simultaneous requests.

>>

>> What I have observed in my testing:

>> - Although Windows 7 and Vista allows more than 10 simultaneous SMB

>> write requests (and Windows 2008 allows more than 50 simultaneous

>> write requests), Windows 7 and Windows Vista appear to have problems

>> when 20 or 30 are in flight at one time.

>> - Windows does not enforce a limit on SMB Negotiate Protocol

>> - Windows seems to ignore checks on maximum simultaneous requests on

>> certain handle based operations (SMB writeX in particular) until

>> after the file is closed (the writes don't get an error, but the next

>> QueryPathInfo after the file is closed gets an error).

>>

>> I need to understand on which requests (besides SMB Negotiate) I

>> should ignore MaxMpxCount.  In particular, SMB Echo, seems like an

>> obvious choice to ALWAYS allow to be sent by the client, since SMB

>> Echo is sent to make sure the server is alive (when for example the

>> limit on simultaneous requests has been reached due to slow opens, or

>> writes past end of file) and to prevent timeout.   If a client is

>> restricted from sending SMB Echo then there is no reasonably reliable

>> mechanism available to determine whether the server is dead or just

>> hung temporarily processing slow requests.

>>

>> MS-CIFS is not clear on which commands MaxMpxCount is enforced.

>>

>> I would like to ignore MaxMpxCount on (sending one of) SMBEcho (where

>> you could cause data integrity issues if you limited the ability to

>> check server state up/down) and SMBNegotiate (where it is obviously

>> not set yet) and SMBOplockBreak responses (since you can't guarantee

>> the order that these are received/sent from the server relative to

>> other frames which are being processed, and can cause the server

>> session to drop if you don't allow the client to send these) to the

>> server since there is no reasonable mechanism to limit these without

>> risking problems with prematurely taking down a slow session (perhaps

>> opens of offline files for example).

>>

>> It looks like the safest way to handle this is for the client to

>> limit pending requests to the MaxMpxCount, with the exception of the

>> three SMB types listed above.

>>

>> In previous versions of MS-CIFS there were mentions of different

>> processes, and uids on the negotiated tcp session and their relation

>> to the MaxMpxCount - and I would also like to verify whether the

>> limits are considered relative to some combination of the negotiated

>> tcp session and uid and/or pid as earlier discussions implied (or

>> file handle, as our testing hinted at).

>>

>>

>>

>>

>>

>> --

>> Thanks,

>>

>> Steve

>>

>

>

>

--

"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X

Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel

jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.

ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org<mailto:crh at ubiqx.mn.org>

OnLineBook -- http://ubiqx.org/cifs/    -)-----   crh at ubiqx.org<mailto:crh at ubiqx.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/cifs-protocol/attachments/20111026/5ef622cc/attachment-0001.html>