[Samba] Question on how to get Samba to use larger pread/write calls.

[Volker Lendecke]

On Mon, Jan 17, 2011 at 10:14:04AM -0600, Chris Duffy wrote:
> We are testing Samba 3 (and 4) on Fedora Core 13,
> 10Gbit connection with a Mac OS 10.6.4 system
> as the client.  We will be adding some Windows
> machines sooner or later with 10Gbit interfaces.
> 
> We are seeing 100-150MBytes/sec read or write
> performance between the Mac and the FC13 system
> over 10Gbit interface but it should be capable of
> 400-500MBytes/sec.  We have a local raid
> on the FC13 system that runs 1GByte/sec locally
> using an Areca 1880-ix-16 raid card (6Gbit version).
> It has 16 fast Hitachi disks in a Raid5 format
> using xfs filesystem.
> 
> The problem here is that samba is poking the Areca
> at 128KByte I/O's on preads and writes, i.e.
> shown to us using strace on the smbd daemons
> that are running.  Using vmstat/iostat/sar utilities,
> we see 100% utilization of the Areca card because the
> average wait time is real high and the average
> queue length to it is also high......too many
> small I/O's.....
> This is not the case if I run "fio" or "dd" locally
> to/from the Areca's raid using 1-4MByte I/O's.
> I see fast I/O...
> 
> I do not see any way to increase the size of Samba's
> pread/write's in the smb.conf documentation.  I
> am sure it may be just a matter of getting the source
> code and making some changes to allow larger
> sized IO's but........and of course I suppose Windows
> clients may complain but....
> 
> I remember that back in the old Irix days with the
> group of engineers in Australia I worked with,
> we had Samba screaming fast but not sure if they
> tweaked the version of Samba on Irix to do this.
> 
> Can you guys come up with a way for us to allow
> the reads/writes to/from the disks to be tunable
> up to say 4MB in size?

You might want to play with "write cache size". This will
only tune writes though. And it will only work for oplocked
files. I'm not 100% sure that OS/X plays nicely wrt oplocks.

For reads, we need to take a much closer look at your real
workload and see if we need to use some kind of preopen,
prefetch or so module. We need to closely work with the
kernel buffer cache and potential readahead kernel
algorithms.

With best regards,

Volker Lendecke

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen