Status and demo of SMB3 Multi-Channel in Samba

[Jeremy Allison]

On Wed, Oct 29, 2014 at 10:52:20PM -0700, Linda W wrote:
> Jeremy Allison wrote:
> >Did you set the "aio read size" or "aio write size" parameters
> >in your smb.conf ?
> >
> >That allows smbd to use a thread per outstanding read or write
> >request which should use multiple cores, depending on how hard
> >the client is driving the server.
> >
> >Jeremy.
> ---
> Didn't notice an appreciable difference nor an extra thread (monitoring with
> htop to show user threads -- which are hilighted.
> 
> Tried with:
>    aio read size = 65536
>    aio write size = 65536
> (as I was using a 16M io/size, I didn't try other aio sizes).

What client are you using to drive the server ?

Setting these value to 65536 is too high. Just
set both to 1 and have done with it.

> Tried with those on the test shares, as well
> as stacking on "vfs_aio_thread" at the end of the vfs chain --
> neither seemed to spawn off extra threads -- both had the
> smb process in the mid 90% usage range for 1 core.

What that's telling you is that IO to and from the
system buffer cache isn't the bottleneck.

If it were you'd see multiple threads waiting on
the IO.

If you're driving the server appropriately you
really should see multiple threads at least on
read requests as each read request coming in will
spawn a thread to do pread().

You should be able to flood the TCP channels with
hundreds (maybe thousands :-) of SMB2 read requests
that will cause threads to stack up.

Writes maybe not so much as the time taken to
do the memcpy from the TCP buffer into the in-memory
buffer, and then from the in-memory buffer into
the buffer cache (and on Linux it's not waiting
until the write hits the metal underneath) might
dominate so any parallelism you get from writes
might not be shown.

How are you driving the server ? What tool are
you using ?