[Samba] Samba async performance - bottleneck or bug?

[douxevip]

Hi David,

> You're still using direct I/O with fio, which will likely disallow
> client side caching with oplocks/leases.

Is there a way to bypass this with settings in smb.conf at all and transform all writes to async?

> I'd recommend checking that your (cifs.ko?) client is using a relatively
> modern SMB2+ dialect and that leases are enabled on both sides.

Yes, I currently have these additional settings in smb.conf:

server min protocol = SMB3_11
client min protocol = SMB3_11
strict sync = no


The issue is that ZFS cannot even do sync writes with this dataset. Every single write is async. But when writing to the Samba share, these speeds do not translate. This might just be the SMB protocol bottleneck - but I'm not sure. If anybody wants to run the same fio tests I did with an async-only pool, it would be interesting to see if it indeed is an SMB bottleneck, or if I'm missing settings in smb.conf.




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, July 25, 2019 2:50 PM, David Disseldorp via samba <samba at lists.samba.org> wrote:

> Hi,
>
> On Fri, 19 Jul 2019 23:26:55 +0000, douxevip wrote:
>
> > So to summarize, this is the situation:
> >
> > 1.  I run a fio benchmark requesting, small, random, async writes. Command is "fio --direct=1 --sync=0 --rw=randwrite --bs=4K --numjobs=1 --iodepth=1 --runtime=60 --group_reporting --name=sambatest --size=32k --time_based". I run this command on both the host, as on the Samba client, both on the same exact ZFS dataset
> >
> > 2.  The ZFS dataset only writes async, converting sync to async writes at all times
> >
> > 3.  That same dataset being shared through Samba, also only performs async writes (strict sync = no)
> >
> > 4.  With these settings, on the host the benchmarks tops out at 520MB/s. On the Samba client (again, writing to the same end destination) I get 40 MB/s tops. I feel like those benchmark scores should be much closer given the above test.
> >
> >
> > I've ruled out the network being an issue since a) they're hosted on the same host via a Linux bridge (which means it never leaves the box) and b)an iperf3 test with a single thread completed the benchmark at 44Gbit/s.
> > I guess that leaves one question: where is the bottleneck here, and is there perhaps another method to speed up these small writes for the sake of experimentation?
>
> You're still using direct I/O with fio, which will likely disallow
> client side caching with oplocks/leases.
> I'd recommend checking that your (cifs.ko?) client is using a relatively
> modern SMB2+ dialect and that leases are enabled on both sides. I
> unfortunately don't have time ATM to analyse this further.
>
> Cheers, David
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba