[cifs-protocol] MS-SWN Q9: Section 3.2.4.27-3.2.4.29 seems to actions triggered when the client receives an RESP_ASYNC_NOTIFY - TrackingID#2311070040010334
Jeff McCashland (He/him)
jeffm at microsoft.com
Wed Jan 3 18:48:31 UTC 2024
Hi Stefan,
Could you let me know when you think you'll have time to provide more information as requested below?
Best regards,
Jeff McCashland (He/him) | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada)
Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300
-----Original Message-----
From: Jeff McCashland (He/him)
Sent: Monday, December 18, 2023 2:49 PM
To: metze <metze at samba.org>
Cc: cifs-protocol at lists.samba.org; Microsoft Support <supportmail at microsoft.com>
Subject: RE: MS-SWN Q9: Section 3.2.4.27-3.2.4.29 seems to actions triggered when the client receives an RESP_ASYNC_NOTIFY - TrackingID#2311070040010334
Hi Stefan,
I didn't see a response to my previous request. It's not clear to us what you are looking for here. Having a single netname for multiple nodes sounds similar to a SOFS configuration. We use DNS to enumerate the IP addresses.
Windows uses witness for the following:
- If networking interface on the server has changed then hint client about that so it can query list of new interfaces sooner than default 10 minutes poling interval.
- If cluster node is down then notify client about that so it can disconnect from the downer and connect to some other node, before TCP/IP timeout expires. Would work only of cluster can detect downer faster than TCP/IP timeout.
- If cluster has asymmetric storage (one node can process IOs faster than the others) then hint client that it should move to that node. In Windows if Direct IO is possible then storage connectivity is considered symmetrical and we prefer load balancing clients across all cluster nodes. If we are in File System Redirected IO (same blog) then storage connectivity is asymmetrical and client is advised to move to the node that has file system mounted to avoid double hop.
All notifications are advisory.
Could you clarify your expectations for the doc and tell us more about what you're trying to accomplish?
Best regards,
Jeff McCashland (He/him) | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada) Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300
-----Original Message-----
From: Jeff McCashland (He/him)
Sent: Tuesday, November 28, 2023 11:00 AM
To: metze <metze at samba.org>
Cc: cifs-protocol at lists.samba.org; Microsoft Support <supportmail at microsoft.com>
Subject: MS-SWN Q9: Section 3.2.4.27-3.2.4.29 seems to actions triggered when the client receives an RESP_ASYNC_NOTIFY - TrackingID#2311070040010334
Hi Stefan,
This is in regards to your question:
Question 9:
Section 3.2.4.27-3.2.4.29 seems to actions triggered when the client receives an RESP_ASYNC_NOTIFY, but there's no specification on how the individual witness registrations handle specific notification events. E.g. based on the different posibilities for RESOURCE_CHANGE.ResourceName Is a CLIENT_MOVE_NOTIFICATION a better choice when using a single InterfaceGroupName for all nodes?
I'm ready to file document change requests to explain the processing, but I don't fully understand your example question. Resource Change notifications are used when resources such as disks change status, while Client Move notifications are used when a node has gone down and the client needs to move to another node.
They aren't interchangeable. Could you clarify your question?
Best regards,
Jeff McCashland (He/him) | Senior Escalation Engineer | Microsoft Protocol Open Specifications Team
Phone: +1 (425) 703-8300 x38300 | Hours: 9am-5pm | Time zone: (UTC-08:00) Pacific Time (US and Canada) Local country phone number found here: http://support.microsoft.com/globalenglish | Extension 1138300
-----Original Message-----
From: Tom Jebo <tomjebo at microsoft.com>
Sent: Tuesday, November 7, 2023 9:47 AM
To: metze <metze at samba.org>
Cc: cifs-protocol at lists.samba.org; Microsoft Support <supportmail at microsoft.com>
Subject: RE: [EXTERNAL] Trying to let a Windows client use MS-SWN against a samba cluster - TrackingID#2311070040009453
[dochelp to bcc]
[support mail to cc]
Hi Stefan,
Thanks for your request regarding MS-SWN (various questions). One of the Open Specifications team members will respond to assist you. In the meantime, we’ve created the following cases to track this request (I've added the first case to the subject so initially, please leave that case number in the subject when communicating with our team about this request.
Q1: 2311070040009453
Q2: 2311070040009696
Q3: 2311070040009793
Q4: 2311070040009927
Q5: 2311070040010003
Q6: 2311070040010094
Q7: 2311070040010182
Q8: 2311070040010257
Q9: 2311070040010334
Q10: 2311070040010406
Q11: 2311070040010486
Best regards,
Tom Jebo
Microsoft Open Specifications Support
-----Original Message-----
From: Stefan Metzmacher <metze at samba.org>
Sent: Tuesday, November 7, 2023 6:30 AM
To: Interoperability Documentation Help <dochelp at microsoft.com>
Cc: cifs-protocol at lists.samba.org
Subject: [EXTERNAL] Trying to let a Windows client use MS-SWN against a samba cluster
Hi DocHelp,
I'm currently implementing MS-SWN for samba in order to allow clients to move to a different network interface or cluster node if a specific interface or a complete cluster node gets offline.
In a Samba cluster we have multiple nodes, but just a single netname for all of them, so there's only a single computer with it's sAMAccountName in active directory. But each node can have multiple ip addresses, which may move around between nodes, but some can be node local.
Now my goal is to let a Windows client use the witness service in order to get notified about ip addresses going down, because the interface link or a whole node gets offline.
In order to archive that I need to understand the exact client behavior implemented in the Windows clients (also with possible differences of various Windows versions).
However this is hard from just reading the existing docs...
MS-SWN "3.2 Witness Client Details" doesn't contain any detail for the logical processing, e.g.
- 3.2.4.1 Application Requests Witness Register
doesn't say that WITNESS_INTERFACE_INFO.InterfaceGroupName
is that name used as part of the servicePrincipalName
(after being prefixed by 'CIFS/') passed to the authentication
layer (spnego, kerberos, ntlm), but I'm seeing this
behavior from a Windows 2022 server as client.
In older version (Windows 2012) I saw that the principal
was requested by the method from MS-RPCE 2.2.1.3.4
rpc_mgmt_inq_princ_name.
Question 1:
Can you please update this with a product behavior note reflecting the reality with all Windows versions.
- 3.2.4.2 Application Requests Witness Event Notification
only says: ...
The status and any received RESP_ASYNC_NOTIFY result obtained from
the server in the previous step MUST be returned to the caller.
- 3.2.4.3 Application Requests Witness UnRegister
Has the following notable section:
... or if the WitnessRegistration.WitnessNotifyRequest is TRUE,
the client MUST stop processing and return an implementation-defined
local error to the caller.
So it seems with a pending AsyncNotify request the Unregister
seems to be skipped.
With that I'd expect the core logic/behavior of a Windows client being specified in MS-SMB2, when I look there I found the following
3.2.5.2 Receiving an SMB2 NEGOTIATE Response
...
If SMB2_GLOBAL_CAP_PERSISTENT_HANDLES is set in the Capabilities field of the
SMB2 NEGOTIATE Response, the client SHOULD invoke the event as specified in
[MS-SWN] section 3.2.4.1 by providing Connection.ServerName as Netname
parameter.
Question 2:
I don't see this happening from a Windows Server 2022 acting as client.
Can you please update this with a product behavior note reflecting the reality with all Windows versions.
3.2.5.5 Receiving an SMB2 TREE_CONNECT Response
...
- TreeConnect.IsCAShare MUST be set to TRUE, if the
SMB2_SHARE_CAP_CONTINUOUS_AVAILABILITY bit is set in the Capabilities
field of the response.
...
If Connection.Dialect belongs to the SMB 3.x dialect family and the Capabilities
field in the response includes SMB2_SHARE_CAP_CLUSTER bit, the client SHOULD
invoke the event as specified in [MS-SWN] section 3.2.4.1 by providing
Connection.ServerName as Netname parameter.
...
If Connection.Dialect belongs to the SMB 3.x dialect family and the Capabilities
field in the response includes the SMB2_SHARE_CAP_SCALEOUT bit, the client MUST
set TreeConnect.IsScaleoutShare to TRUE.
...
If Connection.Dialect is "3.0.2" or "3.1.1" and the Capabilities field in the
response includes the SMB2_SHARE_CAP_ASYMMETRIC bit, the client MUST verify
whether both of the following conditions are true:
...
If the SMB2 TREE_CONNECT request is successful, the client SHOULD invoke the
event as specified in [MS-SWN] section 3.2.4.1 by providing
Connection.ServerName as the Netname parameter and TreeConnect.ShareName as the
ShareName parameter, and by setting the IsShareNameNotificationRequired
parameter to TRUE.
Question 3:
I don't see this happening from a Windows Server 2022 acting as client.
The only relevant flags in order to let the client try a witness connection are SMB2_SHARE_CAP_CONTINUOUS_AVAILABILITY together with SMB2_SHARE_CAP_CLUSTER.
Can you please update this with a product behavior note reflecting the reality with all Windows versions.
3.2.5.6 Receiving an SMB2 TREE_DISCONNECT Response
... If Connection.Dialect belongs to the SMB 3.x dialect family and if
Session.TreeConnectTable is empty in all sessions in the Connection.SessionTable
for which Connection.ServerName matches the server name, the client SHOULD
invoke the event as specified in [MS-SWN] section 3.2.4.3.
Question 4:
I don't see this happening from a Windows Server 2022 acting as client.
The witness registration stays until a reboot.
There's also no new witness registration after a reconnect to a different ip, which means that the smb connection and witness connection may stay on the same server ip address, which means there's no benefit from it.
Can you please update this with a product behavior note reflecting the reality with all Windows versions.
3.2.7.1 Handling a Network Disconnect
...
If Connection.Dialect belongs to the SMB 3.x dialect family and if
Session.TreeConnectTable is empty in all sessions in the Connection.SessionTable
for which Connection.ServerName matches the server name, the client SHOULD
invoke the event as specified in [MS-SWN] section 3.2.4.3.
Question 5:
I don't see this happening from a Windows Server 2022 acting as client.
The witness registration stays until a reboot.
There's also no new witness registration after a reconnect to a different ip, which means that the smb connection and witness connection may stay on the same server ip address, which means there's no benefit from it.
Can you please update this with a product behavior note reflecting the reality with all Windows versions.
3.2.4.3.5 Application Requests Creating a File Opened for Durable Operation
...
- If TreeConnect.IsCAShare is TRUE, the client MUST set the
SMB2_DHANDLE_FLAG_PERSISTENT bit in the Flags field. Otherwise, the client SHOULD
perform one of the following:
- Request a batch oplock by setting RequestedOplockLevel in the create request to
SMB2_OPLOCK_LEVEL_BATCH.
- Request a handle caching lease by including an SMB2_CREATE_REQUEST_LEASE or
SMB2_CREATE_REQUEST_LEASE_V2 Create Context in the create request with a
LeaseState that includes SMB2_LEASE_HANDLE_CACHING.
...
Question 6:
From the documentation the above is the only reference that is impacted by SMB2_SHARE_CAP_CONTINUOUS_AVAILABILITY and it seems that it implicitly disables handle caching, is that really true?
And SMB2_DHANDLE_FLAG_PERSISTENT doesn't seem to be impacted by SMB2_GLOBAL_CAP_PERSISTENT_HANDLES, which is strange.
Can you please cross-check this?
3.2.4.27 Application Notifies Offline Status of a Server
This optional interface is applicable only for the SMB 3.x dialect family. The
application provides the following:
- ServerName: The name of the server which became unavailable.
For each Connection in the ConnectionTable where Connection.ServerName matches
ServerName, the client MUST determine if any TreeConnect exists in the
Session.TreeConnectTable with TreeConnect.IsScaleoutShare set to TRUE.
If a tree connect entry is found, the client MUST do the following:
- Disconnect the connection by performing the steps as specified in section 3.2.7.1.
- Invoke the event as specified in section 3.2.4.28 with ServerName set to the
caller-supplied ServerName.
If no tree connect entry is found, the client MUST disconnect the connection
by performing the steps as specified in section 3.2.7.1.
Question 7:
The above section is the only place in the whole documentation that references SMB2_SHARE_CAP_SCALEOUT, is that really correct?
And it seems to be required in order trigger an immediate reconnect.
Can you please cross-check this?
3.2.4.28 Application Notifies Online Status of a Server
This optional interface is applicable only for the SMB 3.x dialect family. The
application provides the following:
- ServerName: The name of the server which became available.
For each Open in the GlobalFileTable, where Open.Session.ChannelList is empty
and the server name identified from Open.FileName matches ServerName, the client
MUST re-establish the durable open as specified in section 3.2.4.4.
3.2.4.29 Application Requests Moving to a Server Instance
This optional interface is applicable only for SMB 3.x dialect family. The
application provides the following:
- ServerName: The name of the server.
- NewServerAddress: The IPv4 or IPv6 address of the server which the client
is required to move to.
For each Connection in the ConnectionTable where Connection.ServerName matches
ServerName, the client MUST disconnect the connection by performing the steps as
specified in section 3.2.7.1.
For each Open in the GlobalFileTable, where Open.Session.ChannelList is empty
and the server name identified from Open.FileName matches ServerName, the client
MUST re-establish the durable open as specified in section 3.2.4.4, and by using
NewServerAddress as the TransportIdentifier for the rules specified in section
3.2.4.2.
Question 8:
The impact of SMB2_SHARE_CAP_CONTINUOUS_AVAILABILITY
without SMB2_GLOBAL_CAP_PERSISTENT_HANDLES is a very important part of this.
Question 9:
Section 3.2.4.27-3.2.4.29 seems to actions triggered when the client receives an RESP_ASYNC_NOTIFY, but there's no specification on how the individual witness registrations handle specific notification events. E.g. based on the different posibilities for RESOURCE_CHANGE.ResourceName Is a CLIENT_MOVE_NOTIFICATION a better choice when using a single InterfaceGroupName for all nodes?
Question 10:
MS-SWM 3.1.6.1 Server Application Notifies of an Interface Being Enabled or Disabled
The calling application provides the interface group name, IPv4 and/or IPv6
addresses, and state.
...
Then for each entry in the WitnessRegistrationList where
WitnessRegistration.NetworkName
matches the application-provided interface group name ...
This seems to indicate that there's actually just a single InterfaceGroupName matching the single NetworkName.
Question 11:
I'm also wondering if ServerGlobalName is really a single name, as I can the client can use a dns or netbios name of the server!
Thanks!
metze
More information about the cifs-protocol
mailing list