[Samba] samba 3 performance issues

[Greg Folkert]

On Thu, 2006-03-30 at 13:52 -0500, Rohit Kumar Mehta wrote:
> I believe I have some hardware or configuration related performance 
> issues running samba 3.0.14a-3sarge.
> 
> Our server is an Intel Celeron 2 Ghz with 512 MB of RAM and a 3ware
> card using SATA disks in a RAID 5 configuration (3ware controller card). 
>   We have a gigabit network and are using Intel Gigabit ethernet cards 
> e1000).
> 
> When copying large files to the samba shares on the system, the transfer
> rate maxes out near 100 mb/s.  We tested with nttcp and were able to get
> speeds of nearly 800mb/s.  So I think it is safe to conclude this is not 
> a network issue.
> 
> Various tools like top, xosview and mpstat convinced us that we are 
> bound in the CPU. Stopping the samba file transfer and the cpu idle time 
> exceeds 90%.  We are convinced that our CPU is the bottleneck,
> but not sure why.
> 
> #cat /proc/cpuinfo
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 15
> model           : 2
> model name      : Intel(R) Celeron(R) CPU 2.00GHz
> stepping        : 9
> cpu MHz         : 1996.920
> cache size      : 128 KB
> fdiv_bug        : no
> hlt_bug         : no
> f00f_bug        : no
> coma_bug        : no
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 2
> wp              : yes
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid
> bogomips        : 3956.73
> 
> Does anyone have any advice on how to speed up our file transfers? We 
> regularly have to 18 GB worth of files to this system, and it would be 
> very good if we could speed it up.  At current speeds, we get no 
> advantage at all from even having gigabit network cards!
> 
> Please feel free to ask me any other questions about our system setup. 
> Thanks in advance for any advice,

Have you done *ANY* system caching parameters, filesystem tuning, or
Samba Config tuning?

What have you done besides verify it is not the network itself?

Have you tested throughput for the 3ware card?

I can tell you this, if you have the RAID-5 setup not-optimally to work
with the block sizing on your Filesystem you'll never get excellent
throughput.

I always tend to use largest blocking factors with the 3ware cards for
RAID-5. This (for me at least) has proven the fastest and least latency
ridden settings for me. But then I am using XFS on all of my 3ware
RAID-5 setups.

For Mirroring, I typically let the defaults work. Defaults have been by
far the best setup for most filesystems. If you still believe you are
suffering from CPU overload, I'd suggest sending it to the RAID-5 array
with over compressed scp (with mild compression of 4 or 5) and then
without compression. See what you get.

I am betting the real problem comes from multiple bus-mastering cards
conflicting or colliding. The Intel-E1000 and the 3ware card are
definitely both bus-mastering.

There are a couple of things on the Samba side you can do. Turn off
Logging (you don't need it really), change the read and send buffer
sizes, change the TCP setting it uses to be more in line with Gigabit,
move to using Jumbo frames, get a TOE (TCP Offload Engine) NIC.

Then if you still have issues, turn on logging for the stuff you are
worried about (auth would be 0, etc...) and then add a sniffer to you
connection. You'll definitely find something. My gut reaction is that
since this is a Celeron Processor, you really need to goto 64-bit slots
on the mother board. Getting a PCI-X capable motherboard would greatly
help your problems.

One last thing, any of the 95xx cards from 3ware are 3.3V only and are
PCI2.3 compliant, they will function incorrectly possibly even be ruined
or not recognized by a 5V or Auto-detect 5v/3.3v slot. The 9xxx, 8xxx
and 7xxx cards can be used in either a 5V or 3.3V PCI slot. 

Good luck.
-- 
greg, greg at gregfolkert.net

The technology that is 
Stronger, Better, Faster: Linux

Use Debian GNU/Linux, its a bazaar thing

NOTICE: Due to Presidential Executive Orders, the 
National Security Agency may have read this email 
without warning, warrant, or notice, and certainly 
without probable cause. They may do this without 
any judicial or legislative oversight. You have no 
recourse nor protection.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba/attachments/20060330/1205727a/attachment.bin