[Samba] Multichannel and RSS Not Detected

Sat Aug 7 22:43:03 UTC 2021

I've spent the last couple of days trying to get Multichannel and RSS working
and it's been hit and miss.  Mostly miss at this point.

I have two servers on the same network, running the same dual Mellanox 100G
card with mostly identical samba configs.  A Win 10 client sees multichannel
and RSS from one server but not the other when running
"Get-SmbMultichannelConnection -IncludeNotSelected".

Servers are running Devuan (Debian 10) with the 5.10 kernel and ZFS.  I
thought my original issue was because the mlx5_core driver in the 5.10 kernel
did not show any queues in /proc/interrupts (grep lan2 /proc/interrupts). 
Installing the driver from Mellanox fixed that problem.  But the odd thing is,
the server that is working, is running the stock kernel driver and and does
not show anything in /proc/interrupts.  I tried both the stock and Mellanox
driver on the server that is not working.

I'm at a loss on how to debug this on why the Win 10 client does not see
multichannel or RSS for one server but not the other..  I upgraded the server
to samba 4.14.6+dfsg-0.1buster1 from 4.9.  The server that is working used
both versions of samba and the Win 10 client saw it with each version.

I'm mostly intested in getting RSS working as I read for multiple nics to work
I would need to put the IPs on different subnets.

Each server has both of the 100G links bonded into a 802.3ad LAG.

Information:

smb.conf:
[global]
    netbios name = SB1
    interfaces = "172.28.202.10;capability=RSS" "172.28.90.10;capability=RSS"
"172.28.133.10;capability=RSS,speed=100000000000"
        server multi channel support = yes
    workgroup = GIANT
    wins server = 172.28.202.5
    os level = 20
        log level = 3
    client min protocol = SMB3
        os level = 8
        local master = No
        domain master = No
    ntlm auth = yes
    log file = /var/log/samba/log.%m
        max log size = 1000
    server role = standalone server
    obey pam restrictions = No
    unix password sync = no
    passwd program = /usr/bin/passwd %u
    passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:*
%n\n *password\supdated\ssuccessfully*
    invalid users = root
    usershare path =
    usershare allow guests = yes
    guest account = nas
    aio read size = 1
    aio write size = 1
    aio max threads = 100
    vfs objects = catia
        mangled names = no
        catia:mappings =
0x22:0xa8,0x2a:0xa4,0x2f:0xf8,0x3a:0xf7,0x3c:0xab,0x3e:0xbb,0x3f:0xbf,0x5c:0xff,0x7c:0xa6
    smb2 leases = yes
    posix locking = no
    security = user
    map to guest = bad user   
    store dos attributes = yes
    dos filetimes = yes

[plaid]
    path = /netshares/plaid
    force user = nas
    force group = nas
    create mask = 770
    force create mode = 770
    directory mask = 2770
    force directory mode = 2770
    guest ok = yes
    read only = no
    hosts allow = 172.28.133. 172.28.90.
    dos filetimes = yes

ethtool -l lan2
Channel parameters for lan2:
Pre-set maximums:
RX:        0
TX:        0
Other:        0
Combined:    32
Current hardware settings:
RX:        0
TX:        0
Other:        0
Combined:    32

ethtool -g lan2
Ring parameters for lan2:
Pre-set maximums:
RX:        8192
RX Mini:    0
RX Jumbo:    0
TX:        8192
Current hardware settings:
RX:        1024
RX Mini:    0
RX Jumbo:    0
TX:        1024

ethtool -x lan2
RX flow hash indirection table for lan2 with 32 RX ring(s):
    0:      0     1     2     3     4     5     6     7
    8:      8     9    10    11    12    13    14    15
   16:     16    17    18    19    20    21    22    23
   24:     24    25    26    27    28    29    30    31
   32:      0     1     2     3     4     5     6     7
   40:      8     9    10    11    12    13    14    15
   48:     16    17    18    19    20    21    22    23
   56:     24    25    26    27    28    29    30    31
   64:      0     1     2     3     4     5     6     7
   72:      8     9    10    11    12    13    14    15
   80:     16    17    18    19    20    21    22    23
   88:     24    25    26    27    28    29    30    31
   96:      0     1     2     3     4     5     6     7
  104:      8     9    10    11    12    13    14    15
  112:     16    17    18    19    20    21    22    23
  120:     24    25    26    27    28    29    30    31
RSS hash key:
12:e3:f3:8e:92:fa:db:eb:5b:29:e2:49:cd:39:39:9c:a4:ea:41:e8:c9:ee:66:7f:37:9b:ea:33:99:d9:45:3c:8a:db:bd:a1:bc:66:24:05
RSS hash function:
    toeplitz: on
    xor: off
    crc32: off

grep lan2 /proc/interrupts  
    http://ix.io/3viC  (note, need very wide window)

ifconfig bond_private
bond_private: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 9000
        inet 172.28.133.10  netmask 255.255.255.0  broadcast 172.28.133.255
        inet6 fe80::ae1f:6bff:fecf:20e0  prefixlen 64  scopeid 0x20<link>
        ether ac:1f:6b:cf:20:e0  txqueuelen 1000  (Ethernet)
        RX packets 54063955  bytes 326509018851 (304.0 GiB)
        RX errors 0  dropped 5  overruns 0  frame 0
        TX packets 34841728  bytes 299047275167 (278.5 GiB)
        TX errors 0  dropped 5 overruns 0  carrier 0  collisions 0

PS C:\> Get-SmbClientNetworkInterface

Interface Index RSS Capable RDMA Capable Speed   
IpAddresses                                 Friendly Name
--------------- ----------- ------------ -----   
-----------                                 -------------
14              True        False        100 Gbps {fe80::f0b7:6e80:9de4:df83,
172.28.133.101} Ethernet 4

Please let me know if there is any other information I can provide to help me
get this figured out.

    Chris