Segfault in smbd: messaging_dgm_fde_active
Andrew Bartlett
abartlet at samba.org
Mon Jun 25 05:07:36 UTC 2018
G'Day,
Can someone look into this for me please?
The libsmbclient testsuite is triggering this segfault in the messaging
code occasionally:
Here is a Gitlab CI build:
https://gitlab.com/catalyst-samba/samba/-/jobs/76658562
and more importantly on sn-devel here:
https://git.samba.org/autobuild.flakey.sn-devel-144/2018-06-21-0628/samba-ad-dc.stdout
[172(846)/546 at 11m37s] samba3.libsmbclient(ad_dc)
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
mkdir failed on directory /m/autobuild/fl/b506908/prefix/samba-ad-dc/var/cache: No such file or directory
UNEXPECTED(failure): samba3.libsmbclient.readdirplus(ad_dc)
REASON: Exception: Exception: ../source4/torture/libsmbclient/libsmbclient.c:262: failed to create file 'smb://Administrator:locDCpass1@addc/tmp/test_readdirplus.txt': Software caused connection abort
https://git.samba.org/autobuild.flakey.sn-devel-144/2018-06-21-0628/samba-ad-dc.stderr
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: No locals.
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: #6 <signal handler called>
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: No locals.
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: #7 0x00002ba16ba70cd6 in messaging_dgm_fde_active (fde=0x1e9310e600000000) at ../source3/lib/messages_dgm.c:1739
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: flags = 0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: #8 0x00002ba16ba71183 in msg_dgm_ref_recv (ev=0x55a2f70c67d0, msg=0x7ffc0a65a4f8 "\003\316\021", msg_len=116, fds=0x7ffc0a65a1f0, num_fds=0, private_data=0x0) at ../source3/lib/messages_dgm_ref.c:135
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: active = false
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: r = 0x55a2f7281a98
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: next = 0x2000000000000
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: #9 0x00002ba16ba6fc18 in messaging_dgm_recv (ctx=0x55a2f72b63c0, ev=0x55a2f70c67d0, buf=0x7ffc0a65a4f8 "\003\316\021", buflen=116, fds=0x7ffc0a65a1f0, num_fds=0) at ../source3/lib/messages_dgm.c:1333
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: hdr = {msglen = 0, pid = 174433456, sock = 32764}
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: msg = 0x0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: space = 140720482918896
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: cookie = 0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: #10 0x00002ba16ba6fa4b in messaging_dgm_read_handler (ev=0x55a2f70c67d0, fde=0x55a2f821d420, flags=1, private_data=0x55a2f72b63c0) at ../source3/lib/messages_dgm.c:1298
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: num_fds = 0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: i = 0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: fds = 0x7ffc0a65a1f0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: ctx = 0x55a2f72b63c0
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: received = 124
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: msg = {msg_name = 0x0, msg_namelen = 0, msg_iov = 0x7ffc0a65a4a0, msg_iovlen = 1, msg_control = 0x7ffc0a65a200, msg_controllen = 0, msg_flags = 0}
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: iov = {iov_base = 0x7ffc0a65a4f0, iov_len = 1024}
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: msgbufsize = 528
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: msgbuf = 0x7ffc0a65a200 ""
/memdisk/autobuild/fl/b506908/samba-ad-dc/bin/smbd: buf = "\000\000\000\000\000\000\000\000\003\316\021\000\000\000\000\000ӯN*\377\377\377\377>\334ƶ&\275\001\223`8\r\000\000\000\000\000\000\000\000\000\377\377\377\377\226o\205\061c[Zq\002\a\000\000,\031\t\277`\355(I\235\377\320\327\274\260>\330\001\000\000\000\024\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\004", '\000' <repeats 11 times>, "\006", '\000' <repeats 11 times>, "d\000\000\300", '\000' <repeats 844 times>...
I can locally reproduce with Valgrind:
/data/samba/git/samba9/bin/smbd: ==3562== Invalid read of size 8
/data/samba/git/samba9/bin/smbd: ==3562== at 0xAFB0D61: msg_dgm_ref_recv (messages_dgm_ref.c:135)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xAFAF826: messaging_dgm_recv (messages_dgm.c:1333)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xAFAF65C: messaging_dgm_read_handler (messages_dgm.c:1298)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D565D0: epoll_event_loop (tevent_epoll.c:728)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D56C08: epoll_event_loop_once (tevent_epoll.c:930)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D53904: std_event_loop_once (tevent_standard.c:114)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D1E0: _tevent_loop_once (tevent.c:725)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D4F7: tevent_common_loop_wait (tevent.c:848)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D539A6: std_event_loop_wait (tevent_standard.c:145)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D59A: _tevent_loop_wait (tevent.c:867)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F0EE18: smbd_process (process.c:4130)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11499D: smbd_accept_connection (server.c:1031)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D565D0: epoll_event_loop (tevent_epoll.c:728)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D56C08: epoll_event_loop_once (tevent_epoll.c:930)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D53904: std_event_loop_once (tevent_standard.c:114)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D1E0: _tevent_loop_once (tevent.c:725)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D4F7: tevent_common_loop_wait (tevent.c:848)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D539A6: std_event_loop_wait (tevent_standard.c:145)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D59A: _tevent_loop_wait (tevent.c:867)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11569B: smbd_parent_loop (server.c:1383)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x1178C4: main (server.c:2153)
/data/samba/git/samba9/bin/smbd: ==3562== Address 0x27e16400 is 2,112 bytes inside a block of size 8,320 free'd
/data/samba/git/samba9/bin/smbd: ==3562== at 0x4C2CDDB: free (vg_replace_malloc.c:530)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693E962: _tc_free_poolmem (talloc.c:1059)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693EDCB: _tc_free_internal (talloc.c:1194)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693FDF2: _tc_free_children_internal (talloc.c:1646)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693ECF8: _tc_free_internal (talloc.c:1163)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693FDF2: _tc_free_children_internal (talloc.c:1646)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693ECF8: _tc_free_internal (talloc.c:1163)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693FDF2: _tc_free_children_internal (talloc.c:1646)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693ECF8: _tc_free_internal (talloc.c:1163)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693EF54: _talloc_free_internal (talloc.c:1227)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x694021C: _talloc_free (talloc.c:1769)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F31EEC: smbd_smb2_auth_generic_return (smb2_sesssetup.c:469)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F336A7: smbd_smb2_session_setup_auth_return (smb2_sesssetup.c:1005)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F33478: smbd_smb2_session_setup_gensec_done (smb2_sesssetup.c:951)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3C4160: gensec_update_done (gensec.c:517)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3B5E50: gensec_spnego_update_post (spnego.c:2070)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3B59BE: gensec_spnego_update_done (spnego.c:1951)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3C4160: gensec_update_done (gensec.c:517)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3B8DC3: gensec_ntlmssp_update_done (ntlmssp.c:244)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xE3BF3E7: ntlmssp_server_auth_done (ntlmssp_server.c:862)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11AD7AA2: auth_check_password_wrapper_done (auth.c:559)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11AD74E0: auth_check_password_next (auth.c:399)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11AD763D: auth_check_password_done (auth.c:423)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0F2: _tevent_req_error (tevent_req.c:180)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6F5D057: _tevent_req_nterror (tevent_ntstatus.c:46)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11AD9BCC: winbind_check_password_done (auth_winbind.c:226)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x15E3742B: dcerpc_winbind_SamLogon_r_done (ndr_winbind_c.c:4502)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xEA65088: dcerpc_binding_handle_call_done (binding_handle.c:520)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== by 0xEA645CB: dcerpc_binding_handle_raw_call_done (binding_handle.c:203)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4EFAF: _tevent_req_notify_callback (tevent_req.c:125)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F084: tevent_req_finish (tevent_req.c:162)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4F0AC: _tevent_req_done (tevent_req.c:168)
/data/samba/git/samba9/bin/smbd: ==3562== Block was alloc'd at
/data/samba/git/samba9/bin/smbd: ==3562== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693DF3A: __talloc_with_prefix (talloc.c:763)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693E0FA: _talloc_pool (talloc.c:817)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x693E197: talloc_pool (talloc.c:839)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5AB1B1F: talloc_stackframe_internal (talloc_stack.c:180)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5AB1C35: _talloc_stackframe_pool (talloc_stack.c:213)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F0D232: smbd_tevent_trace_callback (process.c:3655)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D9CC: tevent_trace_point_callback (tevent_debug.c:117)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D1C5: _tevent_loop_once (tevent.c:724)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D4F7: tevent_common_loop_wait (tevent.c:848)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D539A6: std_event_loop_wait (tevent_standard.c:145)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D59A: _tevent_loop_wait (tevent.c:867)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x5F0EE18: smbd_process (process.c:4130)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11499D: smbd_accept_connection (server.c:1031)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D565D0: epoll_event_loop (tevent_epoll.c:728)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D56C08: epoll_event_loop_once (tevent_epoll.c:930)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D53904: std_event_loop_once (tevent_standard.c:114)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D1E0: _tevent_loop_once (tevent.c:725)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D4F7: tevent_common_loop_wait (tevent.c:848)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D539A6: std_event_loop_wait (tevent_standard.c:145)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x6D4D59A: _tevent_loop_wait (tevent.c:867)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x11569B: smbd_parent_loop (server.c:1383)
/data/samba/git/samba9/bin/smbd: ==3562== by 0x1178C4: main (server.c:2153)
/data/samba/git/samba9/bin/smbd: ==3562==
Reproduce with:
SAMBA_VALGRIND='valgrind --num-callers=64 --trace-children=yes' make test TESTS=libsmbclient
And the attached patches (which show it isn't somehow the recent dsdb audit code as I had feared).
Tested on current master:
commit 05b54cc259645f69e14de2703724c284ed25838c
Author: Volker Lendecke <vl at samba.org>
Date: Fri Jun 22 16:25:10 2018 +0200
talloc_stack: Call talloc destructors while frame is still around
Any assistance in chasing this down most appreciated.
Thanks,
Andrew Bartlett
--
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team https://samba.org
Samba Development and Support, Catalyst IT
https://catalyst.net.nz/services/samba
-------------- next part --------------
A non-text attachment was scrubbed...
Name: msg-dgm-crash-repo.patch
Type: text/x-patch
Size: 2895 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180625/278b0565/msg-dgm-crash-repo.bin>
More information about the samba-technical
mailing list