[Samba] Transfer rates faster than 23MBps?

[Mark Smith]

Doug VanLeuven wrote:
> Mark Smith wrote:
>> I also tried your values, with the tcp_window_scaling, with no luck.
> It's enable by default, but I explicitly set options other options 
> depend on.

Reasonable idea.  :)

> I set up my test rig again.
> Host server
> 2.6.12-1.1376_FC3, samba 3.0.23
> Broadcom Nextreme BCM5702X Gigabit, tg3 driver default config
> Client
> 2.6.12-1.1381_FC3, samba 3.0.21pre3-SVN-build-11739
> Intel Pro/1000, 82546GB Gigabit, e1000 driver default config
> HD Drives on both are 45-50MBps
> 
> smbclient 26.7-27.2MBps
> ftp 25.4 MBps (small window size)

Yeah, see, that's the difference.  With FTP and HTTP, I'm seeing the 
~60MBps numbers, but SMB is still down at about 22MBps, not even the 
27MBps you're seeing.

> FWIW - I'm used to seeing CIFS performance numbers 5-10% slower than ftp.

5-10% wouldn't surprise me, but 70% slower disturbs me.

> Using ethereal to capture the start of the transfers, I'm seeing windows 
> ftp negotiate a 256960 window size, which is what I have specified in 
> HKLM/system/currcontrolset/services/tcpip/parameters/TcpWindowSize, but 
> linux samba establishes a window size of whatever is specified for 
> SO_SNDBUF in socket options or by default 8K.  So I set SO_SNDBUF=256960 
> and it gave me the extra large window and raised the speed up to 
> 27.3MBps (1048576 Megs) - not enough to really address your concerns.  

Actually, setting SNDBUF and RCVBUF to 65536 from the default of 8192 is 
what got me _TO_ 22MBps...

...Ya know, I once tried increasing SNDBUF and RCVBUF to 256k but didn't 
see any difference.  I've also tried setting the kernel parameters to 
256k, but never both at the same time.  Let me try that and see if it helps.

> Maybe it would be different on your system.  That's an issue for samba 
> because it should allow for autonegotiation of the window size and I 
> don't know how to set that other than ipv4.tcp_window_scaling=1 (the 
> default).  SO_SNDBUF & SO_RCVBUF are only limited by the /proc/sys 
> values* *net.core.rmem_max and net.core.wmem_max which you altered after 
> the earlier post.

See above.  I've set both, but never at the same time.  Let me try that.

> Comparing the linux ftp to linux samba transfer speeds, I don't think 
> the answer lies in samba per se other than how the socket gets set up.  

I thought this too, but Ethereal shows that the Windows client is ACKing 
the TCP stream after only a couple or three packets, no where near the 
32k Window size that is negotiated.  So if Windows were delaying the 
ACK, Linux would still be sending more packets to it.  The sent packets 
are roughly evenly spaced in time, and we're getting them ACKed every 
two or three packets.  It really doesn't look like a TCP Window size 
problem.  (This was the very first path I went down.)

> And it's not a linux issue either if you're getting those http numbers 
> (I never see anything like that here).  Your Redhat is obviously tuned 
> for those types of packets.  Maybe you using the in-kernel optimized 
> apache they offer.  If so, try a user space apache for comparison.

Nope.  Stock Apache 2 as distributed with RedHat AS4-U3.

> I smacked up against these numbers 2 years ago.  Nothing much seems to 
> have changed.  The numbers end up in the low to mid 200Mbps on copper 
> Gigabit for user space applications.  If you ever fix it, pop me an 
> email please.  I figured the answer would be pci-x and 64 bit pci.  
> Higher front side bus speeds.

I will definitely post whatever solution I find here.  Thanks for your 
help, Doug.

-Mark