[Samba] slow throughput on 1gbit lan

[Tomasz Lutelmowski]

Hello,

I'm trying to squeeze out maximum possible transfer between two Samba 3.3.1
servers. Both servers and switch supports jumbo frames, network link and
disks does not appear to be bottleneck, between these two servers FTP
transfer rates are about 115MB/s (1gbit link is saturated at this point) -
and I'd like to see similar transfer from Samba (now I get half of it).

10.0.0.5 # wget ftp://10.0.0.2/1G
100%[=================================================================================================================>]
1,048,576,000  118M/s   in 8.5s
2009-02-27 09:16:00 (118 MB/s) - `1G' saved [1048576000]


Also testing samba locally, it gives me transfer via loopback of over
300MB/s

10.0.0.2 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET 1GB'
WARNING: The "write cache size" option is deprecated
Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
getting file \1GB of size 1048576000 as 1GB (331177.2 kb/s) (average
331177.2 kb/s)


Now, when I try to get this file from another server (10.0.0.5), maximum
transfer drops to 55MB/s....

10.0.0.5 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET 1GB'
Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
getting file \1GB of size 1048576000 as 1GB (55664.3 kb/s) (average 55664.3
kb/s)


So far, I was turning these knobs:

socket options = TCP_NODELAY SO_RCVBUF=262144 SO_SNDBUF=262144
max xmit = 131072
block size = 4096
aio read size = 1
aio write size = 1
use mmap = No
use sendfile = Yes
blocking locks = No

For me only SO_RCVBUF and SO_SNDBUF seems to have any influence, but setting
it to more than 65534 doesn't give any more performance. I suppose TCP
kernel settings are fine, because FTP transfer is able to saturate link...
Do you have any ideas how to improve performance ?

Regards,
Tomek