dcerpc.bare test is a bit flappy

Gary Lockyer gary at catalyst.net.nz
Tue May 21 04:02:05 UTC 2019


I'll take a look at this, probably tomorrow. But running with
AddressSanitizer enabled I get
=================================================================
==10844==ERROR: AddressSanitizer: heap-use-after-free on address
0x612000008920 at pc 0x7f4c173d7a81 bp 0x7ffd153bdea0 sp 0x7ffd153bde90
READ of size 8 at 0x612000008920 thread T0
    #0 0x7f4c173d7a80 in _tevent_schedule_immediate
../../lib/tevent/tevent.c:670
    #1 0x7f4c173dbefc in tevent_req_post ../../lib/tevent/tevent_req.c:257
    #2 0x7f4c120c0a0e in tstream_bsd_disconnect_send
../../lib/tsocket/tsocket_bsd.c:2074
    #3 0x7f4c120badab in tstream_disconnect_send
../../lib/tsocket/tsocket.c:771
    #4 0x7f4c14d84a8a in dcerpc_shutdown_pipe
../../source4/librpc/rpc/dcerpc.c:2367
    #5 0x7f4c14d84a8a in dcerpc_connection_dead
../../source4/librpc/rpc/dcerpc.c:872
    #6 0x7f4c14d84e53 in dcerpc_connection_destructor
../../source4/librpc/rpc/dcerpc.c:118
    #7 0x7f4c17ee80e7 in _tc_free_internal ../../lib/talloc/talloc.c:1157
    #8 0x7f4c17ee80e7 in _tc_free_children_internal
../../lib/talloc/talloc.c:1666
    #9 0x7f4c17ee856d in _tc_free_internal ../../lib/talloc/talloc.c:1183
    #10 0x7f4c17ee856d in _tc_free_children_internal
../../lib/talloc/talloc.c:1666
    #11 0x7f4c17eed0b5 in _tc_free_internal ../../lib/talloc/talloc.c:1183
    #12 0x7f4c17eed0b5 in _talloc_free_internal
../../lib/talloc/talloc.c:1247
    #13 0x7f4c17eed0b5 in _talloc_free ../../lib/talloc/talloc.c:1789
    #14 0x7f4c063db898 in dcerpc_interface_dealloc
../../source4/librpc/rpc/pyrpc.c:322
    #15 0x5023b4  (/usr/bin/python3.6+0x5023b4)
    #16 0x502f3c  (/usr/bin/python3.6+0x502f3c)
    #17 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #18 0x504c27  (/usr/bin/python3.6+0x504c27)
    #19 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #20 0x591460  (/usr/bin/python3.6+0x591460)
    #21 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #22 0x507c16 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x507c16)
    #23 0x504c27  (/usr/bin/python3.6+0x504c27)
    #24 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #25 0x591460  (/usr/bin/python3.6+0x591460)
    #26 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #27 0x54d4e1  (/usr/bin/python3.6+0x54d4e1)
    #28 0x5a730b in _PyObject_FastCallKeywords (/usr/bin/python3.6+0x5a730b)
    #29 0x503072  (/usr/bin/python3.6+0x503072)
    #30 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #31 0x504c27  (/usr/bin/python3.6+0x504c27)
    #32 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #33 0x591460  (/usr/bin/python3.6+0x591460)
    #34 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #35 0x507c16 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x507c16)
    #36 0x504c27  (/usr/bin/python3.6+0x504c27)
    #37 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #38 0x591460  (/usr/bin/python3.6+0x591460)
    #39 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #40 0x54d4e1  (/usr/bin/python3.6+0x54d4e1)
    #41 0x5a730b in _PyObject_FastCallKeywords (/usr/bin/python3.6+0x5a730b)
    #42 0x503072  (/usr/bin/python3.6+0x503072)
    #43 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #44 0x504c27  (/usr/bin/python3.6+0x504c27)
    #45 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #46 0x591460  (/usr/bin/python3.6+0x591460)
    #47 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #48 0x507c16 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x507c16)
    #49 0x504c27  (/usr/bin/python3.6+0x504c27)
    #50 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #51 0x591460  (/usr/bin/python3.6+0x591460)
    #52 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #53 0x54d4e1  (/usr/bin/python3.6+0x54d4e1)
    #54 0x5a730b in _PyObject_FastCallKeywords (/usr/bin/python3.6+0x5a730b)
    #55 0x503072  (/usr/bin/python3.6+0x503072)
    #56 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #57 0x504c27  (/usr/bin/python3.6+0x504c27)
    #58 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #59 0x591460  (/usr/bin/python3.6+0x591460)
    #60 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #61 0x507c16 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x507c16)
    #62 0x504c27  (/usr/bin/python3.6+0x504c27)
    #63 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #64 0x591460  (/usr/bin/python3.6+0x591460)
    #65 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #66 0x54d4e1  (/usr/bin/python3.6+0x54d4e1)
    #67 0x5a730b in _PyObject_FastCallKeywords (/usr/bin/python3.6+0x5a730b)
    #68 0x503072  (/usr/bin/python3.6+0x503072)
    #69 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #70 0x502208  (/usr/bin/python3.6+0x502208)
    #71 0x502f3c  (/usr/bin/python3.6+0x502f3c)
    #72 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #73 0x502208  (/usr/bin/python3.6+0x502208)
    #74 0x502f3c  (/usr/bin/python3.6+0x502f3c)
    #75 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #76 0x504c27  (/usr/bin/python3.6+0x504c27)
    #77 0x501b2d in _PyFunction_FastCallDict (/usr/bin/python3.6+0x501b2d)
    #78 0x591460  (/usr/bin/python3.6+0x591460)
    #79 0x54b812  (/usr/bin/python3.6+0x54b812)
    #80 0x555420  (/usr/bin/python3.6+0x555420)
    #81 0x5a730b in _PyObject_FastCallKeywords (/usr/bin/python3.6+0x5a730b)
    #82 0x503072  (/usr/bin/python3.6+0x503072)
    #83 0x507640 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x507640)
    #84 0x504c27  (/usr/bin/python3.6+0x504c27)
    #85 0x511ec9  (/usr/bin/python3.6+0x511ec9)
    #86 0x502d6e  (/usr/bin/python3.6+0x502d6e)
    #87 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #88 0x504c27  (/usr/bin/python3.6+0x504c27)
    #89 0x50253f  (/usr/bin/python3.6+0x50253f)
    #90 0x502f3c  (/usr/bin/python3.6+0x502f3c)
    #91 0x506858 in _PyEval_EvalFrameDefault (/usr/bin/python3.6+0x506858)
    #92 0x504c27  (/usr/bin/python3.6+0x504c27)
    #93 0x58659c  (/usr/bin/python3.6+0x58659c)
    #94 0x59ebbd in PyObject_Call (/usr/bin/python3.6+0x59ebbd)
    #95 0x63835a  (/usr/bin/python3.6+0x63835a)
    #96 0x639027 in Py_Main (/usr/bin/python3.6+0x639027)
    #97 0x4a6f0f in main (/usr/bin/python3.6+0x4a6f0f)
    #98 0x7f4c1c66eb96 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #99 0x5afa09 in _start (/usr/bin/python3.6+0x5afa09)

0x612000008920 is located 96 bytes inside of 312-byte region
[0x6120000088c0,0x6120000089f8)
freed by thread T0 here:
    #0 0x7f4c1d38fb40 in free
(/usr/lib/x86_64-linux-gnu/libasan.so.5+0xedb40)
    #1 0x7f4c17ee88ab in _tc_free_internal ../../lib/talloc/talloc.c:1221
    #2 0x7f4c17ee88ab in _tc_free_children_internal
../../lib/talloc/talloc.c:1666
    #3 0x7f4c17eed0b5 in _tc_free_internal ../../lib/talloc/talloc.c:1183
    #4 0x7f4c17eed0b5 in _talloc_free_internal
../../lib/talloc/talloc.c:1247
    #5 0x7f4c17eed0b5 in _talloc_free ../../lib/talloc/talloc.c:1789
    #6 0x7f4c063db898 in dcerpc_interface_dealloc
../../source4/librpc/rpc/pyrpc.c:322
    #7 0x5023b4  (/usr/bin/python3.6+0x5023b4)

previously allocated by thread T0 here:
    #0 0x7f4c1d38ff00 in __interceptor_malloc
(/usr/lib/x86_64-linux-gnu/libasan.so.5+0xedf00)
    #1 0x7f4c17ef532c in __talloc_with_prefix ../../lib/talloc/talloc.c:782
    #2 0x7f4c17ef532c in __talloc ../../lib/talloc/talloc.c:824
    #3 0x7f4c17ef532c in _talloc_named_const ../../lib/talloc/talloc.c:981
    #4 0x7f4c17ef532c in _talloc_zero ../../lib/talloc/talloc.c:2422
    #5 0x7f4c173d5e87 in tevent_context_init_ops
../../lib/tevent/tevent.c:487
    #6 0x7f4c173d5f59 in tevent_context_init_byname
../../lib/tevent/tevent.c:523
    #7 0x7f4c146d0977 in s4_event_context_init
../../source4/lib/events/tevent_s4.c:34
    #8 0x7f4c154007a0 in py_dcerpc_interface_init_helper
../../source4/librpc/rpc/pyrpc_util.c:222
    #9 0x7f4c063da5ba in dcerpc_interface_new
../../source4/librpc/rpc/pyrpc.c:388
    #10 0x5553b4  (/usr/bin/python3.6+0x5553b4)

SUMMARY: AddressSanitizer: heap-use-after-free
../../lib/tevent/tevent.c:670 in _tevent_schedule_immediate
Shadow bytes around the buggy address:
  0x0c247fff90d0: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
  0x0c247fff90e0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c247fff90f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c247fff9100: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa
  0x0c247fff9110: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c247fff9120: fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd
  0x0c247fff9130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c247fff9140: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c247fff9150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c247fff9160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa
  0x0c247fff9170: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==10844==ABORTING


On 21/05/19 15:45, Tim Beale wrote:
> Hi Gary,
> 
> I noticed the dcerpc.bare test is a bit flappy. I've seen it fail both
> in gitlab CI and running it locally. Usually it just gives me an error like:
> 
> error:
> samba.tests.dcerpc.bare.samba.tests.dcerpc.bare.BareTestCase.test_two_contexts_tcp
> (samba.subunit.RemotedTestCase)(ad_dc_default:local) [
> Exception: was started but never finished!
> ]
> 
> Sometimes the python test itself produces a segmentation fault.
> 
> It seems like it might be related to commit d65b7641c84976c543d 's4
> librpc rpc pyrpc: Ensure tevent_context deleted last'. On master, the
> test fails maybe 1 in 4 times I run it. I reverted this commit and ran
> it 20+ times without problem.
> 
> To run the test locally, use:
> SELFTEST_TESTENV=ad_dc_default:local make testenv
> python3 -m samba.subunit.run  $LOADLIST samba.tests.dcerpc.bare 2>&1  |
> python3 /home/timbeale/code/samba/selftest/filter-subunit
> --fail-on-empty --prefix="samba.tests.dcerpc.bare."
> --suffix="(ad_dc_default:local)"
> 
> Cheers,
> Tim
> 



More information about the samba-technical mailing list