[Samba] SMB throughput inquiry, Jeremy, and James' bow tie

[Volker Lendecke]

On Tue, Jul 30, 2013 at 02:26:42AM -0500, Stan Hoeppner wrote:
> I went to the site to subscribe again and ended up watching some of
> Jeremy's Google interviews.  I particularly enjoyed the interview with
> James and the bow tie lesson at the end. :)
> 
> So anyway, I recently upgraded my home network to end-to-end GbE.  My
> clients are Windows XP SP3 w/hot fixes, and my Samba server is 3.5.6
> atop vanilla kernel.org Linux 3.2.6 and Debian 6.0.6.
> 
> With FDX fast ethernet steady SMB throughput was ~8.5MB/s.  FTP and HTTP
> throughput were ~11.5MB/s.  With GbE steady SMB throughput is ~23MB/s,
> nearly a 3x improvement, making large file copies such as ISOs much
> speedier.  However ProFTPd and Lighttpd throughput are both a steady
> ~48MB/s, just over double the SMB throughput.
> 
> I've tweaked the various Windows TCP stack registry settings,
> WindowScaling ON, Timestamps OFF, 256KB TcpWindowSize, etc.  Between two
> Windows machines SMB throughput is ~45MB/s.  You can see from the
> remarks below the various smb.conf options I've tried.  No tweaking thus
> far of either Windows or Samba has yielded any improvement, at all.  It
> seems that regardless of tweaking I'm stuck at ~23MB/s.
> 
> [global]
> # max xmit=65536
> # socket options=TCP_NODELAY IPTOS_LOWDELAY
> # read raw=yes
> # large readwrite=yes
> # aio read size=8192
> nt acl support=no
> fstype=Samba
> client signing=disabled
> smb encrypt=disabled
> # smb ports=139
> smb ports=445
> 
> The Linux server has an Intel PRO/1000GT NIC, the clients motherboard
> embedded RealTek 8111/8169, the latter being the reason I'm limited to
> ~50MB/s over the wire.
> 
> I run nmbd via the standard init script at startup but I run smbd via
> inetd.  This doesn't appear to affect throughput.  I effect config
> changes with kill -HUP of inetd and killing smbd.
> 
> I have Wireshark installed on one of the Windows XP machines, though I'm
> a complete novice with it.  I assume a packet trace may be necessary to
> figure out where the SMB request/reply latency is hiding.
> 
> ~23MB/s is a marked improvement and I'm not intending to complain here.
>  It just seems rather low given FTP/HTTP throughput.  I'm wondering how
> much of that ~48MB/s I'm leaving on the table, that could be coaxed out
> of Windows or smbd, the kernel, etc with some tweaking.

The main question is -- does your client issue multiple
requests in parallel? If not, you are effectively limited to
a TCP Window size of roughly 60k, because the higher level
only issues requests of that size sequentially. If you have
a properly multi-threaded or async copy program on the
client, I think even XP would be able to do multi-issue.
With newer clients like Windows 7 the situation is even
better: The SMB2 client is a lot better performance-wise
than XP ever was.

With best regards,

Volker Lendecke

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt at sernet.de