smbd: High CPU usage

[Scott Lovenberg]

> On Jul 19, 2018, at 00:32, Volker Lendecke via samba-technical <samba-technical at lists.samba.org> wrote:
> 
>> On Wed, Jul 18, 2018 at 04:16:46PM -0700, Jeremy Allison via samba-technical wrote:
>>> On Wed, Jul 18, 2018 at 09:20:10PM +0300, Vadim Lazovskiy via samba-technical wrote:
>>> I would not agree.
>>> sendfile is great. It makes you able to serve content directly from disk to
>>> network without unnessesery copying to userspace. It saves you lots
>>> syscalls and context switching.
>>> As for our workload profile (1-2Gbit per server) it really saves CPU time.
>>> 
>>> If someone of samba developers treat this as bug and eager to fix it I
>>> could provide extra information and do some debugging.
>> 
>> Here is a full patchset that should handle all platforms
>> for which we support sendfile(). Can you try it out and confirm it
>> fixes your issue ?
> 
> I know syscalls are cheap, but don't the additional fcntl ones kill
> the performance advantage somewhat? We had discussions with a
> MIPS-based vendor in the past. On that platform (don't remember the
> exact OS version), syscalls were pretty expensive.
> 

It’s been a long time since I’ve spent any time at the hardware level, but isn’t the high cost of system calls on MIPS due to the exaggerated cost of a context switch on that platform due to memory amount/distance? Which leads me to the thought, “will a smart compiler (read: not GCC) pipeline or optimize for the platform architecture enough that the most basic code will be easier to optimize than the ‘clever’ code?” 

The game we’re really playing here is “fit in the data and instruction L1 cache and don’t do things that would upset this rule (context switches, locking, unlikely branches, etc)”? That being said, some operations are going to have much worse performance characteristics on some platforms than they would on others, so optimizing for the most common case might be the best we can do.