SMB Direct Implementation via Linux Device Driver for Samba Design Doc, final?

Or Gerlitz ogerlitz at mellanox.com
Tue Sep 3 16:05:04 CEST 2013


On 03/09/2013 12:08, Stefan (metze) Metzmacher wrote:
>> >In that respect, I am not sure to follow your comment on "could
>> >eliminate multiple copies of the data with shared
>> >memory" as I understood the design document you have sent, the intention
>> >is to allocate memory in the kernel
>> >rdma char-device which will be
>> >
>> >1) used for send/recv/rdma
>> >2) mmap-ed to user space
>> >
>> >So where are the copies in this design?
>> >
>> >As for "involve a contextswitch for most if not all RDMA transfers",
>> >indeed, in this architecture you
>> >will have contextswitch between the rdma daemon to the worker processes
>> >vs. user/kernel mode switch
>> >when you have each worker opening the rdma char-device.
> But I guess each context switch to the deamon will include at least one
> additional context into the kernel.

NO. The fast path verb calls, e.g

ibv_post_send (which is used for rdma-write/rdma-read/atomic ops too)
ibv_post_recv
ibv_poll_cq

don't involve context switch into the kernel but rather only 
writing/reading a descriptor
and writing to a doorbell register, both the descriptor and the doorbell 
are mapped to
the process (daemon in that case) memory.

Or.




More information about the samba-technical mailing list