SMB3 and RDMA support

[Michael Adam]

Richard,

On 2014-05-18 at 13:00 -0700, Richard Sharpe wrote:
> On Sun, May 18, 2014 at 11:39 AM, Michael Adam <obnox at samba.org> wrote:
> > We would like to omit that and treat rdma connections like other
> > smbd-child processes, i.e. have main-smbd listen on rdma and fork
> > a child process when a new rdma connection is accepted and
> > fd-pass them to the already existing smbd process for the client.
> 
> I think that this is possible ... see below ...
> 
> > But we assumed up to now that the rdma client libs don't
> > support forking with established connections.  Hence the need
> > for a single daemon (not forking children) to listen for RDMA
> > connections and do the acual rdma traffic with all clients:
> > the rdma-proxy-daemon.
> 
> This is possible, I believe with ibv_fork_init. Since we would only
> perform RDMA requests in the one daemon (the master, under this
> approach) this should work.

The purpose of forking as I intended it would be to take the
established RDMA connection with it and do the RDMA for the
connection from the child daemon.

What would be the benefit of forking if all RDMA traffic
still goes through the main daemon?

> > Forking without fd-passing gives us one rdma-proxy-daemon per
> > rdma-connection. The ideal solution would (or might) be the
> > solution where there is no proxy daemon at all any more and
> > the rdma traffic is done by the initial smbd process for the
> > client.
> 
> I don't understand the comments about one RDMA deamon per smbd.
> I was not suggesting that

No I was not implying that you suggested something.
I was explaining that, when we can fork but can't afterwards
fd-pass the rdma-connection to the smbd process, we are stuck
not with one rdma-proxy per smbd but with one rdma-proxy per
rdma connection.

> and it cannot be supported by the user-level RDMA
> support in Linux or FreeBSD in any case.

Why is that? I don't understand it.

> >> 5. The smbd-d forks an smbd (the smbd-d is really just an smbd
> >> anyhow), and communicates with this child via a UNIX-domain socket
> >> (transfers the SMB requests and responses via this UNIX-domain
> >> socket.)
> >
> > No! The smbd-d does not fork a child. (At least not in the
> > approach we discussed since we assumed that libibverbs can't
> > cope with forking established rdma connections.)
> 
> That was not the intent. The child process would do no RDMA requests,
> and would have been protected by ibv_fork_init().

I think I don't quite understand your thinking yet.
It was definitely my intent to let the child process do RDMA.

Again: What is the value of a fork from the rdma daemon when
the child does not do rdma?...

Cheers - Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20140518/51916c28/attachment.pgp>