smbd segfault during failed oplock break (Re: Problems with Samba 4 Beta 1 and a possible bug that was previously reported)
Andrew Bartlett
abartlet at samba.org
Fri Jun 29 02:20:08 MDT 2012
On Fri, 2012-06-29 at 02:13 -0600, Trever L. Adams wrote:
> On 06/29/2012 12:15 AM, Trever L. Adams wrote:
> > Alright, I still don't believe I have any logs that show much other than
> > the above. And the one below. I have narrowed this a great deal. I have
> > three other machines this should have hit, but the order of turning off
> > offline files and leaving the nonexistant (removed) domain and joining
> > the new S4 provision and then turning back on offline files must have
> > saved the other 3 machines this trouble.
> >
> > So, the problem is that the laptop in question was joined to the old
> > domain, that domain disappeared, offline files were disabled (not sure
> > the order here) and rejoined to the new provision. It appears that the
> > offline files were still cached on the laptop and trying to sync. This
> > was causing problems as user ids, domain sid, etc. had all changed. This
> > seems to be what was causing the crash/hang in smbd. Below is a log from
> > that. This log is from before flushing all offline files and restarting.
> > I haven't yet turned back offline files, but without it everything is
> > working fine.
> >
> > The log is from 4.0.0beta3-GIT-7468ce6
> >
> > [2012/06/28 23:49:04.217451, 0]
> > ../source3/smbd/oplock.c:333(oplock_timeout_handler)
> > Oplock break failed for file admin.V2/NTUSER.DAT -- replying anyway
> > [2012/06/28 23:49:04.223361, 0] ../source3/lib/util.c:863(smb_panic_s3)
> > PANIC (pid 6663): Got a deferred entry without a request: PANIC:
> > share_mode_entry[1]: pid = 6663, share_access = 0x0, private_options =
> > 0x0, access_mask = 0x0, mid = 0x19, type= 0x20, gen_id = 0, uid =
> > 4294967295, flags = 0, file_id fd02:55400ac:0, name_hash = 0x0
> >
> > [2012/06/28 23:49:04.230720, 0] ../source3/lib/util.c:974(log_stack_trace)
> > BACKTRACE: 27 stack frames:
> > #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f)
> > [0x7fd71d0f39f0]
> > #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c)
> > [0x7fd71d0f3852]
> > #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28)
> > [0x7fd71ef00a15]
> > #3 /usr/local/samba/lib/private/libsmbd_base.so(+0x120264)
> > [0x7fd71e6a9264]
> > #4 /usr/local/samba/lib/private/libsmbd_base.so(+0x12057b)
> > [0x7fd71e6a957b]
> > #5 /usr/local/samba/lib/private/libsmbd_base.so(+0x122c65)
> > [0x7fd71e6abc65]
> > #6 /usr/local/samba/lib/private/libsmbd_base.so(+0x12647b)
> > [0x7fd71e6af47b]
> > #7
> > /usr/local/samba/lib/private/libsmbd_base.so(create_file_default+0x2e5)
> > [0x7fd71e6aff87]
> > #8 /usr/local/samba/lib/private/libsmbd_base.so(+0x22f96d)
> > [0x7fd71e7b896d]
> > #9
> > /usr/local/samba/lib/private/libsmbd_base.so(smb_vfs_call_create_file+0xc8)
> > [0x7fd71e6bb0e6]
> > #10 /usr/local/samba/lib/private/libsmbd_base.so(+0x16c158)
> > [0x7fd71e6f5158]
> > #11
> > /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_process_create+0x783)
> > [0x7fd71e6f3327]
> > #12
> > /usr/local/samba/lib/private/libsmbd_base.so(smbd_smb2_request_dispatch+0x6da)
> > [0x7fd71e6eb796]
> > #13 /usr/local/samba/lib/private/libsmbd_base.so(+0x16d0cc)
> > [0x7fd71e6f60cc]
> > #14
> > /usr/local/samba/lib/private/libtevent.so.0(tevent_common_loop_immediate+0x1f5)
> > [0x7fd71d35bca4]
> > #15 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x56)
> > [0x7fd71d10f3d5]
> > #16 /usr/local/samba/lib/libsmbconf.so.0(+0x41c65) [0x7fd71d10fc65]
> > #17
> > /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xe0)
> > [0x7fd71d35ae18]
> > #18 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x10c3)
> > [0x7fd71e6d5f13]
> > #19 /usr/local/samba/sbin/smbd() [0x40949b]
> > #20 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x72b)
> > [0x7fd71d10faaa]
> > #21 /usr/local/samba/lib/libsmbconf.so.0(+0x41d40) [0x7fd71d10fd40]
> > #22
> > /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xe0)
> > [0x7fd71d35ae18]
> > #23 /usr/local/samba/sbin/smbd() [0x40a02b]
> > #24 /usr/local/samba/sbin/smbd(main+0x1468) [0x40b5e4]
> > #25 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fd71b9b1735]
> > #26 /usr/local/samba/sbin/smbd() [0x4052c9]
> > [2012/06/28 23:49:04.237445, 0] ../source3/lib/util.c:875(smb_panic_s3)
> > smb_panic(): calling panic action [/bin/sleep 999999999]
> > [2012/06/28 23:53:41.406470, 0]
> > ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
> > talloc: access after free error - first free may be at
> > ../source3/smbd/server_exit.c:195
> > [2012/06/28 23:53:41.410814, 0]
> > ../source3/lib/popt_common.c:67(popt_s3_talloc_log_fn)
> > Bad talloc magic value - access after free
> > [2012/06/28 23:53:41.410987, 0] ../source3/lib/util.c:863(smb_panic_s3)
> > PANIC (pid 6692): Bad talloc magic value - access after free
> > [2012/06/28 23:53:41.414733, 0] ../source3/lib/util.c:974(log_stack_trace)
> > BACKTRACE: 41 stack frames:
> > #0 /usr/local/samba/lib/libsmbconf.so.0(log_stack_trace+0x1f)
> > [0x7fd71d0f39f0]
> > #1 /usr/local/samba/lib/libsmbconf.so.0(smb_panic_s3+0x6c)
> > [0x7fd71d0f3852]
> > #2 /usr/local/samba/lib/libsamba-util.so.0(smb_panic+0x28)
> > [0x7fd71ef00a15]
> > #3 /usr/local/samba/lib/private/libtalloc.so.2(+0x20ad) [0x7fd71e3830ad]
> > #4 /usr/local/samba/lib/private/libtalloc.so.2(+0x2129) [0x7fd71e383129]
> > #5 /usr/local/samba/lib/private/libtalloc.so.2(+0x21a6) [0x7fd71e3831a6]
> > #6 /usr/local/samba/lib/private/libtalloc.so.2(_talloc_free+0x36)
> > [0x7fd71e385047]
> > #7 /usr/local/samba/lib/private/libsmbd_base.so(+0x167e8f)
> > [0x7fd71e6f0e8f]
> > #8 /usr/local/samba/lib/private/libtalloc.so.2(+0x2cbf) [0x7fd71e383cbf]
> > #9 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #10 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #11 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #12 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #13 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #14 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #15 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #16 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #17 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #18 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #19 /usr/local/samba/lib/private/libtalloc.so.2(+0x3d1a) [0x7fd71e384d1a]
> > #20 /usr/local/samba/lib/private/libtalloc.so.2(+0x2e62) [0x7fd71e383e62]
> > #21 /usr/local/samba/lib/private/libtalloc.so.2(_talloc_free+0x102)
> > [0x7fd71e385113]
> > #22 /usr/local/samba/lib/private/libsmbd_base.so(+0x185188)
> > [0x7fd71e70e188]
> > #23 /usr/local/samba/lib/private/libsmbd_base.so(+0x1853a5)
> > [0x7fd71e70e3a5]
> > #24 /usr/local/samba/sbin/smbd() [0x40821b]
> > #25 /usr/local/samba/lib/libsmbconf.so.0(messaging_dispatch_rec+0x83)
> > [0x7fd71d0fc6d6]
> > #26 /usr/local/samba/lib/libsmbconf.so.0(+0x2fb25) [0x7fd71d0fdb25]
> > #27 /usr/local/samba/lib/libsmbconf.so.0(+0x2e79a) [0x7fd71d0fc79a]
> > #28
> > /usr/local/samba/lib/private/libtevent.so.0(tevent_common_check_signal+0x20a)
> > [0x7fd71d35e9e2]
> > #29 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x2f)
> > [0x7fd71d10f3ae]
> > #30 /usr/local/samba/lib/libsmbconf.so.0(+0x41c65) [0x7fd71d10fc65]
> > #31
> > /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xe0)
> > [0x7fd71d35ae18]
> > #32 /usr/local/samba/lib/private/libsmbd_base.so(smbd_process+0x10c3)
> > [0x7fd71e6d5f13]
> > #33 /usr/local/samba/sbin/smbd() [0x40949b]
> > #34 /usr/local/samba/lib/libsmbconf.so.0(run_events_poll+0x72b)
> > [0x7fd71d10faaa]
> > #35 /usr/local/samba/lib/libsmbconf.so.0(+0x41d40) [0x7fd71d10fd40]
> > #36
> > /usr/local/samba/lib/private/libtevent.so.0(_tevent_loop_once+0xe0)
> > [0x7fd71d35ae18]
> > #37 /usr/local/samba/sbin/smbd() [0x40a02b]
> > #38 /usr/local/samba/sbin/smbd(main+0x1468) [0x40b5e4]
> > #39 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fd71b9b1735]
> > #40 /usr/local/samba/sbin/smbd() [0x4052c9]
> >
> > I hope this will enable you the bug to be fixed. Additionally, I am
> > still seeing the DNS problem mentioned in this thread and under the
> > thread titled "Problems (possibly bug) with dlz for bind 9.9 in
> > 4.0.0beta3-GIT-763f9e8"
> >
> > Thank you,
> > Trever
> >
>
> Hello All,
>
> I think I may have found why things are hanging. By hanging I mean smbd
> seems to stop responding and another process, which may or may not
> respond, is created. So, I go from 1-2 smbd to dozens over time.
>
> I understand that network errors are not something smbd can prevent.
> However, it shouldn't hang/duplicate whenever it gets told a computer
> which is talking to it is not reachable (icmp6 destination unreachable
> in this case).
>
> This is randomly happening. I believe it is a bug in linux's rp_filter
> and/or virtual bridge/virtio io pci device in kvm. I am looking into
> this now.
>
> So, I am not sure what smbd should do, but I believe it is receiving
> these messages (tcpdump sure shows the wifi interface on the router
> sending them and the virtual machine receiving them). I do not believe
> it should hang/duplicate. Log an error and terminate the request?
I've retitled the bug to get the attention of those who work on the file
server, as this isn't a specifically AD related bug as far as I can see.
If you can get it all under valgrind, it may help working out the
details of the use-after-free().
Thanks!
Andrew Bartlett
--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
More information about the samba-technical
mailing list