[Samba] Gigabit Throughput too low

[Greg Freemyer]

On 5/13/05, Duncan, Brian M. <brian.duncan at kattenlaw.com> wrote:
> 
> 512 Megs of ram in server
> Pentium 4 2.0 Ghz
> 2 Ultra 133 controllers - 8 drives total - all 300 gig drives running at
> max UDMA support with write back cache turned off on each drive.
> (Clients connected are all at 1 Gig full duplex)
> 
> FC2, with Samba 3.0.10-1
> 
> Any tweaks I try I test before and after and have only left in place
> what tweaks seemed to improve performance.
> 
> I am just running into a wall with Linux's manner that it handles
> caching of the files I think before it writes them to disk.  I have seen
> my transfers start out as high as 50 Megabytes per second, but then they
> slowly go down (seen it go as low as 1 Megabyte per sec) My guess is if
> I added more memory to the server that time for it to slow down would be
> increased a bit. (I was going to confirm that this weekend)
> 

You do know that 10 MB/sec is not horrible for what you describe above.

ie. You have a very low cost ide-controller structure.  You have
multiple drives per ide channel (in use at the same time?  I hope not
due to master/slave contention).

You don't describe any raid.  raid-10 is typically the fastest way to
go, but uses more drives.  A good 8-drive raid-10 is theoretically 4x
faster than no raid on writes and 8x faster on reads.  (Admittedly,
that is only in theory, but it should still be faster.)

You don't mention the filesystem, but I'm guessing ext3, which is also
not a great speed daemon.  I'm guessing you have the default
journaling setup.  Asssuming a journalling FS, you want to put the
journal on some dedicated spindles, not the same drives as the FS.

Basically, I would optimize your disk-subsystem speed before I started
worrying about your 1Gb/sec. LAN.

Personally, I would consider 3ware parallel IDE controllers, raid 5 at
a minimum (raid 10 if you can), xfs filesystem, dedicated journal
drive.

3ware has a white-paper describing a high-performance Linux setup. 
You might want to look at it.

With a $1000 3ware controller and lots of reconfiguration, you can
probably get your disk sub-system up to 100MB/sec with no probem. 
Even 200 or 300 should be achievable.

Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century