[Samba] Samba performance

[Alan Hodgson]

On Wed, May 25, 2011 at 08:02:56PM -0700, Juan Pablo wrote:
> > - 4 Intel Gigagit ethernet NIC ports with 802.3ad bonding connected to a
> > switch configured tu use 802.3ad
> > - 8 2TB 7.2 krpm SATA disks with hardware RAID5 (RAID stripe size 1024
> > bytes, controller and disk cache enabled, readahead enabled)
> > - XFS filesystem (created with the following parameters: size=64k -d
> > su=1024k,sw=7)
> > - Average file size in the share: 8 MByte
> > - Gigabit network composed by Cat5E certified cabling and DLink DGS-3427
> > gigabit switch.

The way Linux does 803.ad is not really how you might expect.

Basically, it isn't really smart about how it assigns individual network 
streams to ports. It hashes destination MAC addresses and then randomly 
assigns them to ports. It doesn't pay attention to current traffic levels. It is 
quite likely that all 4 connections ended up using the same port - which you 
could see by monitoring the total packets transmitted on each interface during 
your test.

You can make it somewhat more likely to use more ports by using a different 
hashing algorithm. I've had the best results with something like this in 
modprobe.conf:

"options bonding mode=802.3ad miimon=100 lacp_rate=1 
xmit_hash_policy=layer2+3"

It's still not great though. You'd really be better off with a 10Gb/s interface 
out to your switch if you need to guarantee multiple 1Gb/s connections over a 
small number of simultaneous connections.

-- 
The whole universe is change and life itself is but what you deem it.