[Samba] SMB Multichannel not working?

Wed Feb 1 18:49:07 UTC 2023

Hello,

I, like many before me, am having an issue with SMB multichannel. I am running Samba v4.10.16 (package is provided by Amazon) on an Amazon Linux 2 EC2 instance. I am aware that SMB multichannel is an experimental feature until 4.15, but the documentation states it is supported so I would like to evaluate it.

We have had reports of performance degradation when certain workloads (heavy write workloads) are using this Samba server. Upon further investigation, I believe we are not getting the benefits of SMB multichannel. According to the documentation the feature can be enabled by setting "server multi channel support = true" and also an option to force it on. There are also many articles or posts where it is suggested to set interfaces along with their IP, speed, and capabilities (such as RSS). All of these things are set in my configuration, but SMB multichannel does not engage.
On the client side, I am running another EC2 instance but it is a Windows Server 2019 instance. SMB3 is supported and multichannel support is enabled by default, along with BandwidthThrottling being disabled. When I connect to an SMB share hosted on another Windows EC2 instance, the results of the Get-SmbMultichannelConnection -IncludeNotSelected command returns that the connection is a multichannel connection. However, when connected to an SMB share hosted on my Samba EC2 instance, I get no results for it at all. The results of Get-SmbConnection does return a result showing a connection is established to my Samba server.

I have done a LOT of digging including trying a newer Samba version (v4.16.5), which is after 4.15 where multichannel is no longer experimental, and I got the same results. I am aware that SMB multichannel from a hardware perspective relies on multiple physical CPU cores and either multiple NICs or a single RSS-capable NIC on both sides. I have tried EC2 instances (both client and server) that have had up to 16 physical cores each (32 vCPUs with hyperthreading), 32 GB of memory, and NICs capable of 25 Gbps. AWS has concepts of CPU and Network burst credits but for short bursts it should not be a problem here.

I have tried multiple storage types including EBS, EFS, and FSx (ZFS). For EBS, with 16k provisioned IOPS and a 1 GB/s throughput max, I was only getting 300-400 MB/s average and 500-600 MB/s max throughput at the beginning of transfers through samba. No results returned in powershell for multichannel connections. I got very similar performance from the other solutions even when I have configured the resources with a specific performance configuration trying to test peak performance.

My goal here is to ultimately confirm that SMB multichannel is working, but I have a feeling that the results of the PowerShell commands on the client side prove it is not. Thank you for taking the time to read this and if you provide feedback I greatly appreciate it!

Here is my smb.conf file:
[global]
    log file = /var/log/samba/%m
    log level = 1
    unix password sync = yes
    netbios name = sambatoebs
    map to guest = bad user
    guest account = samba-client
    hosts allow = 172.27.252.0/255.255.255.0
    force user = samba-client
    guest ok = yes
    force create mode = 777
    force directory mode = 777
    server multi channel support = yes
    force:server multi channel support = yes
    interfaces = "172.27.252.40;capability=RSS,speed=25000000000"
    aio write size = 1
    aio read size = 1
[filestore]
    path = /mnt/filestore
    read only = no
    writable = yes
    browsable = yes
    public = yes
[filestore_readonly]
    path = /mnt/filestore
    read only = yes
    writable = no
    browsable = yes
    public = yes
[tmp]
    path = /mnt/filestore/tmp
    read only = no
    writable = yes
    browsable = yes
    public = yes

[https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_03.png]
Carter Sheehan
Cloud Engineer
o: +1 (781) 224-7753<tel:+1%20(781)%20224-7753>
e: csheehan at vestmark.com<mailto:csheehan at vestmark.com>
Learn more about our next-generation UMA capabilities - Watch the Video<https://vimeo.com/543224747>
Check us out on Social Media:   [https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_07.png] <https://www.linkedin.com/company/vestmark/>  [https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_09.png] <https://www.youtube.com/channel/UCjiTzgNkrybI3RiI4KUrKGQ/featured>   [https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_11.png] <https://twitter.com/Vestmark>
This e-mail and any attachments hereto, are intended for use by the addressee(s) only and may contain information that is confidential information of Vestmark, Inc. If you are not the intended recipient of this e-mail, or if you have otherwise received this e-mail in error, please immediately notify me by telephone or by e-mail, and please permanently delete the original, any print outs and any copies of the foregoing. Any dissemination, distribution or copying of this e-mail is strictly prohibited.