Netbench results on a linux 4-way system
habanero at us.ibm.com
Wed Jul 31 11:06:02 GMT 2002
I though I'd share some NetBench results on one of our servers.
4 x 1.5 GHz P4, 256K L3, 32MB L4, 2 GB memory
4 x 1Gbps acenic ethernet
14 SCSI disks in hardware RAID1 with 128 MB writeback NVRAM
SuSE 8.0, 2.4.18 kernel
48 x 866 MHz PIII running Windows 2000
Baseline 576 Mbps
ext3 data=writeback 623 Mbps
samba smblog=1 673 Mbps
sendfile/zerocopy 801 Mbps
O(1) scheduler 809 Mbps
Evenly affined IRQs 800 Mbps *needed to get process affinity correct
Process affinity 848 Mbps
/proc/sys/net/hll=764 853 Mbps
case sens enforced 895 Mbps
samba spinlocks 912 Mbps
dcache read copy update 923 Mbps (also had 5% idle time)
I have also achieved 1002 Mbps with ext2.
Some other things I think may be worth investigating:
gettimeofday(). Samba calls this a lot, one for every reply I think, to check
for connection timeout. This means we go into kernel mode every single time
we call this, something I'd like to avoid. And I also don't think we need
the resolution of gettimeofday for this. How about some sort of timer in
samba with a 1 second granularity? I admit I have not thought about how to
do this, but there's gotta be a way.
locking for the samba db. Spin locks got us a little better than flocks, but
again I'd rather not go into kernel mode every time. Has anyone considered
using Rusty Russell's futexes for this?
Hyperthreading. With 2 physical processors, I can get 25% better results!!!
with 4 physical processors, I only get 2% better. I may be running into
other bottlenecks on the 4 physical/8 logical CPU case, so I hope there is
room for improvement. However there are probably a lot more 2-way P4 systems
out there than 4-way, so I bet this could really benefit a lot of people out
Anyway, I have a lot of analysis data (kernel profiles mostly) that goes along
with this stuff if anyone is interested in looking at it My intention was to
make some sort of article out of this, so I wanted to get some feedback from
this list. Is this stuff interesting or relevant to anyone out there? What
else would you like to see in terms of samba on linux performance?
More information about the samba-technical