[Samba] Possible Issue with Samba Blocking I/O and CPU

[Stan Hoeppner]

Andy Liebman put forth on 6/10/2010 9:21 AM:
> Presumably, with this setting, Samba will only
> write out in 256K blocks (which happens to be the stripe size on our RAIDS)


Jeremy Allison put forth on 6/10/2010 12:09 PM:
> "almost never hit the disk" is on a per-client basis :-).
> If a client were mainly reading/writing within a local
> area in a file, that fits within the 256m cache, then it
> will be served mainly out of that cache.


According to Jeremy, using "write cache size = 262144", you end up with a 256
_megabyte_ cache for _each_ smbd process.  Apparently the setting value is in
kilobytes, not bytes.  With 10 active users that equals ~2.5GB of system
(virtual) memory allocated to smbd read/write cache.  This would definitely
explain why you're masking any underlying kernel I/O problem.  As Jeremy said,
with this configuration, you're now rarely touching the disks.

This obviously has nothing to do with matching your RAID stripe width.  You've
apparently unknowingly thrown so much memory buffer at the problem it is
effectively masked.  You can test this by dropping from 262144 to 256,
lowering the per smbd cache from 256MB to 256KB (which will then equal your
stripe width), and your problem will likely resurface.

Until you solve the kernel issue, and if memory consumption is now an issue,
you may want to play with "write cache size =" values starting at 1024 (1MB)
and increasing it 1 or 2MB at a time until the problem is masked again.  Gut
instinct tells me you shouldn't need a whole 256MB of cache per smbd process
to mask the problem.  You may find that 16MB or less per process will do the
trick.  Season to taste.  Please keep us updated.  These kinds of problems
fascinate me, maybe others as well.

-- 
Stan