smbd stuck at writev() on FreeBSD if 'use sendfile = yes' in smb.conf

Youzhong Yang Youzhong.Yang at
Mon Nov 6 00:51:30 UTC 2017

Hi Timur,

That’s too bad. Yes, I am using zfs on freebsd.

This one explains why:

“ZFS currently does not play nicely with sendfile(2). The idea with sendfile(2) is to share pages between the buffer cache and the mbuf, to prevent copying. The call results in a range of bytes from an open file descriptor being faulted into the buffer cache, and then those pages being shared all the way through to the network stack until the bytes are transmitted.

With ZFS, since it does not use the OS buffer cache, data is copied from the ZFS ARC to the buffer cache, where it is then shared with the mbuf etc. Solaris has a solution to solve this issue, and it might make sense to adapt our sendfile mechanism to work in a similar way. [Solaris PSARC: Copy Reduction Interface]( This creates two new VFS operations, reqzcbuf which asks the filesystem to prepare a UIO for zero copy operations. In the case of ZFS, this involves allocating a new array to track the ARC headers of each buffer that is loaned out from the ZFS ARC.”

But it’s really bad that using sendfile() causes Samba to hang.


From: Timur I. Bakeyev [mailto:timur at]
Sent: Sunday, November 05, 2017 7:23 PM
To: Youzhong Yang <Youzhong.Yang at>
Cc: samba-technical at
Subject: Re: smbd stuck at writev() on FreeBSD if 'use sendfile = yes' in smb.conf

Hi, Youzhong!

You didn't specify what file system do you use. Please, keep in mind that ZFS doesn't support 'sendfile' properly.

With regards,
Timur Bakeyev

On Fri, Nov 3, 2017 at 9:28 PM, Youzhong Yang via samba-technical <samba-technical at<mailto:samba-technical at>> wrote:
Sorry, I meant:

if 'use sendfile' is set to no, everything works well. But if it is set to yes, here is what happened.

From: Youzhong Yang
Sent: Friday, November 03, 2017 4:27 PM
To: samba-technical at<mailto:samba-technical at>
Subject: smbd stuck at writev() on FreeBSD if 'use sendfile = yes' in smb.conf

Hi All,

I am struggling with a smbd issue on FreeBSD (11.1). In my smb.conf, if 'use sendfile' is set to yes, everything works well. But if it is set to no, here is what happened:

  *   sendfile() first returns 35 (EAGAIN):

# dtrace -n 'syscall::sendfile:return /arg1 == -1/ {ustack(); printf("errno = %d", errno);}'
dtrace: description 'syscall::sendfile:return ' matched 1 probe
CPU     ID                    FUNCTION:NAME
23  64274                  sendfile:return
errno = 35

  *   then smbd keeps calling writev(), which always returns -1 with errno being EAGAIN:

# dtrace -n 'syscall::writev:return /arg1 == -1/ {ustack(); printf("errno = %d", errno);}'
16  63748                    writev:return
errno = 35

  *   and this is the stack trace of smbd:

# echo bt | lldb -p 1810
(lldb) process attach --pid 1810
Process 1810 stopped

Executable module set to "/tmw-nas-3p/samba/sbin/smbd".
Architecture set to: x86_64--freebsd11.1.
(lldb) bt
* thread #1
  * frame #0: 0x0000000805b4ba2a`_writev + 10
    frame #1:`__thr_writev(fd=<unavailable>, iov=<unavailable>, iovcnt=<unavailable>) at thr_syscalls.c:634
    frame #2:`sys_writev(fd=40, iov=0x00007fffffffcd58, iovcnt=1) at sys_rw.c:101
    frame #3:`write_data_iov(fd=40, orig_iov=0x00007fffffffcd58, iovcnt=1) at sys_rw_data.c:49
    frame #4:`write_data(fd=40, buffer=0x000000081e7ac340, n=65536) at sys_rw_data.c:94
    frame #5:`fake_sendfile(xconn=0x0000000812c74d60, fsp=0x000000081e438ce0, startpos=63438848, nread=839680) at reply.c:3329
    frame #6:`smb2_sendfile_send_data(state=0x000000081e432660) at smb2_read.c:283
    frame #7:`_tc_free_internal(tc=0x000000081e432600, location="../source3/smbd/smb2_server.c:3697") at talloc.c:1078
    frame #8:`_tc_free_children_internal(tc=0x000000081e432080, ptr=0x000000081e4320e0, location="../source3/smbd/smb2_server.c:3697") at talloc.c:1593
    frame #9:`_tc_free_internal(tc=0x000000081e432080, location="../source3/smbd/smb2_server.c:3697") at talloc.c:1104
    frame #10:`_talloc_free_internal(ptr=0x000000081e4320e0, location="../source3/smbd/smb2_server.c:3697") at talloc.c:1174
    frame #11:`_talloc_free(ptr=0x000000081e4320e0, location="../source3/smbd/smb2_server.c:3697") at talloc.c:1716
    frame #12:`smbd_smb2_flush_send_queue(xconn=0x0000000812c74d60) at smb2_server.c:3697
    frame #13:`smbd_smb2_request_reply(req=0x000000081e4320e0) at smb2_server.c:3011
    frame #14:`smbd_smb2_request_done_ex(req=0x000000081e4320e0, status=(v = 0), body=(data = "\x11", length = 16), dyn=0x00007fffffffd680, location="../source3/smbd/smb2_read.c:164") at smb2_server.c:3159
    frame #15:`smbd_smb2_request_read_done(subreq=0x0000000000000000) at smb2_read.c:164
    frame #16:`_tevent_req_notify_callback(req=0x000000081e4324d0, location="../source3/smbd/smb2_server.c:1387") at tevent_req.c:120
    frame #17:`smbd_smb2_request_pending_queue(req=0x000000081e4320e0, subreq=0x000000081e4324d0, defer_time=500) at smb2_server.c:1387
    frame #18:`smbd_smb2_request_process_read(req=0x000000081e4320e0) at smb2_read.c:109
    frame #19:`smbd_smb2_request_dispatch(req=0x000000081e4320e0) at smb2_server.c:2641
    frame #20:`smbd_smb2_io_handler(xconn=0x0000000812c74d60, fde_flags=1) at smb2_server.c:3950
    frame #21:`smbd_smb2_connection_handler(ev=0x0000000812c5a1a0, fde=0x0000000812c4fba0, flags=1, private_data=0x0000000812c74d60) at smb2_server.c:3988
    frame #22:`poll_event_loop_poll(ev=0x0000000812c5a1a0, tvalp=0x00007fffffffe068) at tevent_poll.c:605
    frame #23:`poll_event_loop_once(ev=0x0000000812c5a1a0, location="../source3/smbd/process.c:4140") at tevent_poll.c:662
    frame #24:`_tevent_loop_once(ev=0x0000000812c5a1a0, location="../source3/smbd/process.c:4140") at tevent.c:721
    frame #25:`poll_event_loop_wait(ev=0x0000000812c5a1a0, location="../source3/smbd/process.c:4140") at tevent_poll.c:678
    frame #26:`_tevent_loop_wait(ev=0x0000000812c5a1a0, location="../source3/smbd/process.c:4140") at tevent.c:863
    frame #27:`smbd_process(ev_ctx=0x0000000812c5a1a0, msg_ctx=0x0000000812c4f120, sock_fd=40, interactive=false) at process.c:4140
    frame #28: smbd`smbd_accept_connection(ev=0x0000000812c5a1a0, fde=0x0000000812c4fba0, flags=1, private_data=0x0000000812c428c0) at server.c:1024
    frame #29:`poll_event_loop_poll(ev=0x0000000812c5a1a0, tvalp=0x00007fffffffe548) at tevent_poll.c:605
    frame #30:`poll_event_loop_once(ev=0x0000000812c5a1a0, location="../source3/smbd/server.c:1391") at tevent_poll.c:662
    frame #31:`_tevent_loop_once(ev=0x0000000812c5a1a0, location="../source3/smbd/server.c:1391") at tevent.c:721
    frame #32:`poll_event_loop_wait(ev=0x0000000812c5a1a0, location="../source3/smbd/server.c:1391") at tevent_poll.c:678
    frame #33:`_tevent_loop_wait(ev=0x0000000812c5a1a0, location="../source3/smbd/server.c:1391") at tevent.c:863
    frame #34: smbd`smbd_parent_loop(ev_ctx=0x0000000812c5a1a0, parent=0x0000000812c52680) at server.c:1391
    frame #35: smbd`main(argc=5, argv=0x00007fffffffed08) at server.c:2050
    frame #36: 0x0000000001027cf0 smbd`_start + 384

In my smb.conf, I have socket options set as follows:


By the way, I am running Samba 4.6.8. Has anyone experienced the same issue? Is it a FreeBSD kernel bug?



More information about the samba-technical mailing list