Corrected diagram and description for SMB-Direct (RDMA) support in Samba
realrichardsharpe at gmail.com
Sun May 18 17:29:06 MDT 2014
I have redrawn my diagram and it has gotten more complex. It is attached.
The steps are:
0. The master smbd starts.
1. It does a fork and exec or otherwise starts the smbdirectd. This is
the only process that uses RDMA via librdmacm, libibverbs, etc.
2. A tcp connection from a client arrives at the master smbd.
3. The master smbd forks a child smbd 'c1' to handle the connection.
The child handles all client SMB requests, including, most likely, an
FSCTL to return the interfaces. The response to this would include the
4. An RDMA connection comes in to the smbdirect daemon, perhaps in
response to the client connected to smbd 'c1' finding out that it has
an RDMA interface in common with the server. This is accepted by the
5. When the SMB Direct channel is established and the first SMB
arrives, the smbdirect daemon informs the master smbd and provides it
with the name of a UNIX domain socket to use.
6. The master smbd now forks a new child smbd, 'c2', which uses the
supplied UNIX domain socket to communicate with the smbddirect daemon.
7. The child smbd, c2, now handles the first part of the SMB protocol
over the supplied UNIX domain socket. This includes the
NegotiateProtocol and SessionSetup request. Upon receiving the
SessionSetup request, c2 uses the clientGuid to determine which child
smbd should handle the connection. In this case, it is c1. It might
find this mapping by looking in the connections TDB, for example.
8. The child smbd, c2, now transfers the UNIX domain socket to c1 and exits.
After that, c1 will handle both connections, one directly, and one via
the UNIX domain socket connection it has to the smbdirect daemon.
For those SMB requests that use RDMA READ/WRITE requests (, ie, for
which it has descriptors) it uses a shared-memory area established by
the smbdirect daemon for this purpose.
If Michael and I are now on the same page, we can move on to other issues.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 30456 bytes
Desc: not available
More information about the samba-technical