[Samba] This old tune again : read performance optimisations
Jeremy Allison
jra at samba.org
Thu Mar 22 10:58:31 MDT 2012
On Thu, Mar 22, 2012 at 05:49:44PM +0100, Emmanuel Florac wrote:
> Hi list,
> I have this problem of (relatively) poor samba read performance. Read
> performance is constantly, reproducibly lower than write performance
> and I can't really understand why.
>
> Here is the server configuration :
>
> dual octocore Opteron 6128, 32 GB RAM
> Adaptec 6445 RAID card
> 36x 2TB Hitachi SATA drives (raid 60)
>
> Sustained local disk throughput: over 850 MB/s write, over 1.4 GB/s
> read.
>
> kernel 3.1.10 (but I tried various versions with no more
> success), amd64, pure vanilla kernel.org version.
> Distro Debian squeeze amd64, samba 2:3.5.6~dfsg-3squeeze5
>
> The client is an opteron dual core machine running Win7 64 bits. This is
> by far the best performing CIFS client I've tested, software wise (see
> below).
>
> The systems are connected in 10GigE ethernet using Myricom Myri-10G
> dual ports (only one port used) through a Fujitsu 10GigE switch.
>
> I first tested the same client to the same server using Linux (same
> configuration as the server) and netperf, then NFS, then cifs mount.
>
> Netperf (TCP_SENDFILE test) gives more than 9500 Mb/s both direction, so
> the network hardware works OK.
>
> NFS gives the following results both in UDP and TCP modes:
> Sustained throughput: 490 to 520 MB/s write, 620 to 650 MB/s read.
>
> The linux CIFS client running on the same machine writes at a puny 180
> MB/s and read at a miserable 75 MB/s. OK, never mind, samba is for
> windows clients, right?
>
> Now, the same machine rebooted under windows 7 writes at 450 MB/s,
> which is close enough to NFS performance, but can't top 400 MB/s
> reading. Read performance *should* be higher. I've played with various
> parameters; I can get better write performance at times, on par with
> NFS, but I never at any time remotely get as good a read performance.
>
> I'd really like to get this beast tamed at last :)
>
>
> Here comes the smb.conf file :
>
> [global]
> name resolve order = wins lmhosts host bcast
> socket options = TCP_NODELAY
>
> interfaces = eth2
> bind interfaces only = yes
> aio read size = 1
> aio write size = 1
> aio write behind = yes
Until the next 3.6.x ships with the aio_pthread module,
you might we better off removing the "aio XXX" lines.
Linux glibc aio is fundamentally broken (and Ulrich
doesn't seem interested in fixing it) - this is why
I ended up writing the aio_pthread module.
You could try using Volker's aio_fork module, which
might help (with the Linux clients, but probably not
the Windows clients unless you're running SMB2).
You're you're more adventurous you could try running
from v3-6-test git tree and ensure you have the
aio_pthread module loaded.
Jeremy.
More information about the samba
mailing list