smbd: High CPU usage

[Jeremy Allison]

On Wed, Jul 18, 2018 at 09:20:10PM +0300, Vadim Lazovskiy via samba-technical wrote:
> I would not agree.
> sendfile is great. It makes you able to serve content directly from disk to
> network without unnessesery copying to userspace. It saves you lots
> syscalls and context switching.
> As for our workload profile (1-2Gbit per server) it really saves CPU time.
> 
> If someone of samba developers treat this as bug and eager to fix it I
> could provide extra information and do some debugging.

Need more details. Is this on Linux ? With SMB2 ?

It's probably this (from the Linux part of source3/lib/sendfile.c):

        while (total) {
                ssize_t nwritten;
                do {
                        nwritten = sendfile(tofd, fromfd, &offset, total);
                } while (nwritten == -1 && (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK));

Try bracketing the sendfile with :

set_blocking(tofd, true);

... sendfile call

set_blocking(tofd, false);

We set the send socket to non-blocking by default, but might need
to explicitly change to blocking if sendfile is enabled (then set
back to non-blocking after the sendfile).

Jeremy

> ср, 18 июл. 2018 г. в 17:21, Volker Lendecke <Volker.Lendecke at sernet.de>:
> 
> > On Wed, Jul 18, 2018 at 04:30:06PM +0300, Vadim Lazovskiy via
> > samba-technical wrote:
> > > I've encountered the issue in samba4. I'm totally convinced it is a bug.
> > > In *some* cases with *some* clients smbd process hogs the CPU.
> > > Attaching strace to such process shows huge amount sendfile() syscall
> > > that's trying to write to non-blocking socket while write is actually
> > would
> > > block:
> > >
> > > sendfile(37, 34, [320625896], 238360)   = -1 EAGAIN (Resource temporarily
> > > unavailable)
> > > sendfile(37, 34, [320625896], 238360)   = -1 EAGAIN (Resource temporarily
> > > unavailable)
> > > sendfile(37, 34, [320625896], 238360)   = -1 EAGAIN (Resource temporarily
> > > unavailable)
> > > sendfile(37, 34, [320625896], 238360)   = -1 EAGAIN (Resource temporarily
> > > unavailable)
> > > sendfile(37, 34, [320625896], 238360)   = -1 EAGAIN (Resource temporarily
> > > unavailable)
> > >
> > > Among this flood there are some rare successful writes:
> > > sendfile(37, 34, [320497416] => [320561656], 366840) = 64240
> > >
> > > 37 is the network descriptor
> > > 34 is the file descriptor.
> > >
> > > I've faced this issue several years ago when upgraded samba to 4.0. Then
> > > I've decided to downgrade to 3.5. But now there is no way to stay at 3.x.
> > >
> > > Of course it is possible to completely disable sendfile using "use
> > sendfile
> > > = no", then samba starts to use read and write syscalls but it would show
> > > some performance decline.
> >
> > Unless you are on an embedded platform like ARM or MIPS you should not
> > notice any performance decline. I would suggest disabling sendfile.
> >
> > Volker
> >
> > --
> > SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
> > phone: +49-551-370000-0, fax: +49-551-370000-9
> > AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
> > http://www.sernet.de, mailto:kontakt at sernet.de
> >
> 
> 
> -- 
> WBR,
> Vadim Lazovskiy