[Samba] suggestions for a "fast" fileserver - 1G / 10G

[Emmanuel Florac]

Le Mon, 24 Mar 2014 18:37:34 +0800
Christopher Chan <christopher.chan at bradbury.edu.hk> écrivait:

> If you want to talk about performance, please do use ext3 or ext4
> with a BBU NVRAM block device for the journal and use full journaling
> mode. Not risky filesystems like XFS.

Please. I've set up, shipped, installed and supported thousands of file
servers on XFS, it's time to stop this sort of trolling. I'm a linux
storage professional not a high-schooler playing with an old PC in the
attic.

Re-read my message. On the same hardware (dual opteron, 35 TB array),
right now, running 3.12.7 kernel, default mkfs and mount options (+
nobarrier), XFS is 30% faster than btrfs for single threaded sequential
reads, even faster for concurrent reads; and 20% faster than ext4 at
sequential writes, and even better for concurrent writes. It's just
faster. If what you care about is speed, it's the FS you want.

It may be slightly less safe than ext4 with all the slower options
enabled (data journaling and the like). However for all practical
purposes it's safe enough, it doesn't eat your data, even when pulling
the plug (if you have BBUs...). I know it, because it happened on me
this very morning on a 40 TB server running since 2008.

There have been a troubled period with XFS, but it was something like 8
or 9 years ago (around the time SGI hit the ground). Didn't you notice
RH ships XFS nowadays, and employs many of the maintainers and the
project leader?

> >
> > 4° filesystem options: if you have a RAID controller with BBU or
> > like to live dangerously, use nobarrier option or barrier=0.  Don't
> > use data=ordered (well, if you really want to use ext4 try not to
> > tie its short legs).  
> 
> I'd prefer to use md and a bbu-nvram card than rely on a hardware
> raid controller. If you are doing raid5 or raid6 then make sure you
> have plenty of cache on that controller along with the backup battery.

For any filesystem bigger than say 10 or 12 TB, you should use RAID-6,
because disk bit error rates play against you (unrecoverable bit
error rate is generally one or two orders of magnitude worse than the
brochure values). 

BTW provided you'll probably need to add some SAS ports to your server
and given the very hefty price of NVRAM cards, I think the hardware
RAID is a cheaper option. 

But sure, software RAID is actually faster than hardware RAID, at the
detriment of some CPU power and memory bandwidth. I'd advise against
software RAID on application servers for this reason; there is no "one
size fit all", as always.

All of this is fine and dandy but drifts away from the OP concerns. I'd
insist on the obvious: you can reach very high network performance even
with Samba 3.x.x, given that the underlying filesystem responds really
really fast to small requests. ext3/4 with the default options
definitely won't cut it. Particularly not ext3 which has no real reason
to be used anymore.

I'd even add that network optimisations such as jumbo frames come last.
They practically do  nothing unless you're running on 10GigE, anyway.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac at intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------