[Samba] suggestions for a "fast" fileserver - 1G / 10G

[Jeremy Allison]

On Mon, Mar 24, 2014 at 09:45:38AM +0100, Emmanuel Florac wrote:
> Le Sat, 22 Mar 2014 12:37:15 +0100 vous écriviez:
> 
> > I'm looking for some recommendations for a "fast" fileserver regarding
> > the hardware you use.
> > 
> 
> What is your disk subsystem like? What filesystem, with which mount
> options? Any server I've setup in the past 3 years routinely achieve
> 110 MB/s with samba on GigE, 600 to 800 MB/s using 10 GigE. 
> 
> The very very most important option is :
> 
> 1° don't ever use CFQ on a file server. Use noop or eventually
> deadline. 
> 
> CFQ absolutely sucks wind on servers. CFQ is only good for workstation
> use of rotating disks (it absolutely sucks with SSDs, too). On servers
> with a hardware RAID controller, use noop. Using md raid, noop or
> deadline may be better depending upon your workflow.
> 
> 2° Then comes the hardware: your disk subsystem should be about twice as
> fast as you want your network to be. To achieve smooth, 100 MB/s
> throughput, aim for 200 MB/s disk speed. Any pair of modern drives
> should come close, anyway. Any decent RAID controller (not HP smartshit)
> should achieve 500 GB/s in RAID 6 with 6 or 8 drives.
> 
> 3° the filesystem is important. Don't use ext3, it's slow as snails
> crawling in molasses. ext4 is pretty slow, too.  If you want to get
> closer to bare metal performance, XFS is the best choice unless you
> need more than 25 ACLs per file.
> 
> 4° filesystem options: if you have a RAID controller with BBU or like
> to live dangerously, use nobarrier option or barrier=0.  Don't use
> data=ordered (well, if you really want to use ext4 try not to tie its
> short legs).
> 
> 5° queue, read ahead options are very  important too. Default linux
> options are all thought out for single, 2004-vintage ATA disks. They're
> far from  optimal on modern hardware. They're even farther for RAID
> arrays. Play with the nr_requests and read_ahead_kb options
> from /sys/block/sdXX/queue . On modern RAID arrays, typical value I use
> are
> 
>  cat /sys/block/sda/queue/nr_requests 
> 1024
> 
>  cat /sys/block/sda/queue/read_ahead_kb 
> 32768

Really interesting and valuable insights - thanks !

Maybe we should add these to the wiki ?

Jeremy.