[Samba] Samba async performance - bottleneck or bug?
ddiss at samba.org
Fri Jul 19 12:00:17 UTC 2019
On Thu, 18 Jul 2019 19:04:47 +0000, douxevip via samba wrote:
> I have a ZFS dataset that has sync writes disabled (setting sync=disabled) which means that it will only do async writes, and sync requests get converted to async writes. The ZFS dataset is hosted on a single Samsung 840 Pro 512GB SATA SSD.
> I have this same dataset served as a Samba share, using Proxmox VE 6. Samba version 4.9.5-Debian (Buster), protocol SMB3_11. Kernel version 5.0.15.
> To illustrate, when I do a random sync write benchmark on the host on this dataset, it will use RAM to do the write, drastically speeding up random writes.
> The below benchmark command on the ZFS host:
> fio --direct=1 --sync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --group_reporting --name=sambatest --time_based
> Has an average speed of 520MB/s (which is the maximum speed of my SATA SSD). Despite requesting a sync write, ZFS turns it in an async write, dramatically speeding it up. Clearly the results are great when I directly benchmark from the host into the sync=disabled ZFS dataset. But this doesn't translate to real-world Samba speeds as I would've hoped. My goal is to get Samba as fast as possible with random writes like these in both my Windows 10 Professional PC (1903) as well as a VM on the same Proxmox host, running a clean Debian 10 Buster install.
Hmm, so this "async" (sync=disabled?) ZFS tunable means that it
completely ignores O_SYNC and O_DIRECT and runs the entire workload in
RAM? I know nothing about ZFS, but that sounds like a mighty dangerous
setting for production deployments.
> However, when doing the exact same speedtest command listed above from the Debian 10 VM that has that exact same CIFS share mounted, I only get 35MB/s max speeds. On my Windows 10 machine, using a 700MB folder filled with thousands of files between 16 - 64KBs, I get worse speeds even - hovering around 15MB/s and completing the task in about 60 seconds. To clarify - these are all hosted on the same ZFS dataset, which transforms sync writes into async writes.
> My question is 1) how can I get Samba to write these to RAM (async) for much faster speeds? I thought that setting it up in the ZFS dataset should take care of that, but it does not seem to make any difference.
If you want to test SMB client buffered I/O, then I'd suggest that you
enable oplocks/leases (should be on by default), and then run fio
without the sync / direct parameters. If you want Samba to ignore SMB
client sync requests, then you could set "strict sync = no", but like
the sync=disabled ZFS tunable you mentioned, this parameter plays
russian roulette with your data if the server goes down unexpectedly.
> 2) Where is the bottleneck exactly?
> I currently have this setup in my smb.conf, just listing the lines I edited (the rest is default):
> netbios name = prox
> case sensitive = no
> server min protocol = SMB3
> client min protocol = SMB3
> path = /zfs/synctest
> valid users = myuser
> read only = no
> writeable = yes
> guest ok = no
> create mode = 770
> directory mode = 770
> syncops:disable = true # I tried the test with both this option enabled as well as commented out, and it didn't seem to make a difference.
syncops parameters only have an effect if the syncops VFS module is
More information about the samba