[Samba] SMB throughput inquiry, Jeremy, and James' bow tie

[Stan Hoeppner]

I went to the site to subscribe again and ended up watching some of
Jeremy's Google interviews.  I particularly enjoyed the interview with
James and the bow tie lesson at the end. :)

So anyway, I recently upgraded my home network to end-to-end GbE.  My
clients are Windows XP SP3 w/hot fixes, and my Samba server is 3.5.6
atop vanilla kernel.org Linux 3.2.6 and Debian 6.0.6.

With FDX fast ethernet steady SMB throughput was ~8.5MB/s.  FTP and HTTP
throughput were ~11.5MB/s.  With GbE steady SMB throughput is ~23MB/s,
nearly a 3x improvement, making large file copies such as ISOs much
speedier.  However ProFTPd and Lighttpd throughput are both a steady
~48MB/s, just over double the SMB throughput.

I've tweaked the various Windows TCP stack registry settings,
WindowScaling ON, Timestamps OFF, 256KB TcpWindowSize, etc.  Between two
Windows machines SMB throughput is ~45MB/s.  You can see from the
remarks below the various smb.conf options I've tried.  No tweaking thus
far of either Windows or Samba has yielded any improvement, at all.  It
seems that regardless of tweaking I'm stuck at ~23MB/s.

[global]
# max xmit=65536
# socket options=TCP_NODELAY IPTOS_LOWDELAY
# read raw=yes
# large readwrite=yes
# aio read size=8192
nt acl support=no
fstype=Samba
client signing=disabled
smb encrypt=disabled
# smb ports=139
smb ports=445

The Linux server has an Intel PRO/1000GT NIC, the clients motherboard
embedded RealTek 8111/8169, the latter being the reason I'm limited to
~50MB/s over the wire.

I run nmbd via the standard init script at startup but I run smbd via
inetd.  This doesn't appear to affect throughput.  I effect config
changes with kill -HUP of inetd and killing smbd.

I have Wireshark installed on one of the Windows XP machines, though I'm
a complete novice with it.  I assume a packet trace may be necessary to
figure out where the SMB request/reply latency is hiding.

~23MB/s is a marked improvement and I'm not intending to complain here.
 It just seems rather low given FTP/HTTP throughput.  I'm wondering how
much of that ~48MB/s I'm leaving on the table, that could be coaxed out
of Windows or smbd, the kernel, etc with some tweaking.

I don't want to take up a bunch of anyone's time with this.  If you can
just tell me what information you need in order to point me in the right
direction, I'll do my best to provide it with little fuss.

Thanks again for providing such an invaluable piece of open source
software to the world.

-- 
Stan