[Samba] ctdb error after upgrade to 4.12.10

[Andrew Bartlett]

No.

Debug symbols are additional bits of the Samba software generated by
the compiler and sometimes packages separately by your Samba
distributor (eg SerNet).

Install that and we will get a better backtrace.  Even better if you
can set up the panic action to use our gdb_backtrace script, but that
is a little more complex.

Andrew Bartlett

On Mon, 2020-11-09 at 18:15 +0100, Benedikt Kaleß via samba wrote:
> Dear Jeremy,
> 
> thanks for your reply.
> 
> I assume I load the the debug symbols by passing something in the
> [global] section?
> 
> I now changed back to a fuse_mount to get my colleagues back to work again.
> 
> Tomorrow at 17 CEST I can add a test share with vfs_glusterfs to test it.
> 
> Best
> 
> Bene
> 
> Am 09.11.20 um 17:55 schrieb Jeremy Allison:
> > On Mon, Nov 09, 2020 at 04:01:47PM +0100, Benedikt Kaleß via samba wrote:
> > > Dear List,
> > > 
> > > I updated to samba 4.12.10-SerNet-Debian-9.buster from 4.12.9
> > > and did a
> > > 
> > > gluster volume set volume performance.write-behind off
> > > 
> > > Now I get this in /var/log/syslog and randomly people don't get access
> > > to a shae.
> > 
> > Can you load the debug symbols please ? That should give us
> > more data on where this is failing.
> > 
> > Thanks !
> > 
> > Jeremy.
> > 
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.078120,  0]
> > > ../../source3/lib/popt_common.c:68(popt_s3_talloc_log_fn)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   talloc: access after free error
> > > - first free may be at ../../source3/modules/vfs_glusterfs.c:347
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.078173,  0]
> > > ../../source3/lib/popt_common.c:68(popt_s3_talloc_log_fn)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   Bad talloc magic value - access
> > > after free
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.078184,  0]
> > > ../../source3/lib/util.c:830(smb_panic_s3)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   PANIC (pid 16832): Bad talloc
> > > magic value - access after free
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.078539,  0]
> > > ../../lib/util/fault.c:265(log_stack_trace)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   BACKTRACE: 34 stack frames:
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #0
> > > /usr/lib/x86_64-linux-gnu/samba/libsamba-util.so.0(log_stack_trace+0x30)
> > > [0x7f53811addde]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #1
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbconf.so.0(smb_panic_s3+0x24)
> > > [0x7f5380c2b05f]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #2
> > > /usr/lib/x86_64-linux-gnu/samba/libsamba-util.so.0(smb_panic+0x2c)
> > > [0x7f53811adee1]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #3
> > > /usr/lib/x86_64-linux-gnu/samba/libtalloc.so.2(+0x6f9b) [0x7f5380dcdf9b]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #4
> > > /usr/lib/x86_64-linux-gnu/samba/libtalloc.so.2(+0x6fb2) [0x7f5380dcdfb2]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #5
> > > /usr/lib/x86_64-linux-gnu/samba/libtalloc.so.2(_talloc_free+0x188)
> > > [0x7f5380dcda22]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #6
> > > /usr/lib/x86_64-linux-gnu/samba/vfs/glusterfs.so(+0x645c) [0x7f537b16045c]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #7
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(smb_vfs_call_connect+0x21)
> > > [0x7f5380fdcddd]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #8
> > > /usr/lib/x86_64-linux-gnu/samba/vfs/acl_xattr.so(+0x1765) [0x7f537b156765]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #9
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(smb_vfs_call_connect+0x21)
> > > [0x7f5380fdcddd]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #10
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(+0x1e1e43)
> > > [0x7f5380ff4e43]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #11
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(make_connection_smb2+0x53)
> > > [0x7f5380ff5fac]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #12
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(smbd_smb2_request_process_tcon+0x45c)
> > > [0x7f538100c972]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #13
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(smbd_smb2_request_dispatch+0x12e2)
> > > [0x7f5381004609]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #14
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(+0x1f34f1)
> > > [0x7f53810064f1]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #15
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(tevent_common_invoke_fd_handler+0x84)
> > > [0x7f5380df4dcd]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #16
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xd378) [0x7f5380dfb378]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #17
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xafc8) [0x7f5380df8fc8]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #18
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_once+0xa2)
> > > [0x7f5380df45e0]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #19
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(tevent_common_loop_wait+0x21)
> > > [0x7f5380df47db]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #20
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xaf76) [0x7f5380df8f76]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #21
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_wait+0xa)
> > > [0x7f5380df4837]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #22
> > > /usr/lib/x86_64-linux-gnu/samba/libsmbd-base-samba4.so(smbd_process+0x909)
> > > [0x7f5380ff3b2b]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #23 /usr/sbin/smbd(+0xccc3)
> > > [0x55c2f0fcccc3]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #24
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(tevent_common_invoke_fd_handler+0x84)
> > > [0x7f5380df4dcd]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #25
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xd378) [0x7f5380dfb378]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #26
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xafc8) [0x7f5380df8fc8]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #27
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_once+0xa2)
> > > [0x7f5380df45e0]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #28
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(tevent_common_loop_wait+0x21)
> > > [0x7f5380df47db]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #29
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(+0xaf76) [0x7f5380df8f76]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #30
> > > /usr/lib/x86_64-linux-gnu/samba/libtevent.so.0(_tevent_loop_wait+0xa)
> > > [0x7f5380df4837]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #31
> > > /usr/sbin/smbd(main+0x1d21) [0x55c2f0fceb66]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #32
> > > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f53802b109b]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:    #33
> > > /usr/sbin/smbd(_start+0x2a) [0x55c2f0fc712a]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.078657,  0]
> > > ../../source3/lib/util.c:842(smb_panic_s3)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   smb_panic(): calling panic
> > > action [/usr/share/samba/scripts/panic-action 16832]
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.332923,  0]
> > > ../../source3/lib/util.c:850(smb_panic_s3)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   smb_panic(): action returned
> > > status 0
> > > Nov  9 15:56:14 ctdb-host smbd[16832]: [2020/11/09 15:56:14.332972,  0]
> > > ../../source3/lib/dumpcore.c:315(dump_core)
> > > Nov  9 15:56:14 ctdb-host smbd[16832]:   dumping core in
> > > /var/log/samba/cores/smbd
> > > 
> > > Shall I roll back?
> > > 
> > > Best regards
> > > 
> > > Bene
> > > 
> > > 
> > > 
> > > -- 
> > > forumZFD
> > > Entschieden für Frieden|Committed to Peace
> > > 
> > > Benedikt Kaleß
> > > Leiter Team IT|Head team IT
> > > 
> > > Forum Ziviler Friedensdienst e.V.|Forum Civil Peace Service
> > > Am Kölner Brett 8 | 50825 Köln | Germany  
> > > 
> > > Tel 0221 91273233 | Fax 0221 91273299 | 
> > > http://www.forumZFD.de 
> > > 
> > > Vorstand nach § 26 BGB, einzelvertretungsberechtigt|Executive Board:
> > > Oliver Knabe (Vorsitz|Chair), Sonja Wiekenberg-Mlalandle, Alexander Mauz  
> > > VR 17651 Amtsgericht Köln
> > > 
> > > Spenden|Donations: IBAN DE37 3702 0500 0008 2401 01 BIC BFSWDE33XXX
> > > 
> > > 
> > > -- 
> > > To unsubscribe from this list go to the following URL and read the
> > > instructions:  https://lists.samba.org/mailman/options/samba
> 
> -- 
> forumZFD
> Entschieden für Frieden|Committed to Peace
> 
> Benedikt Kaleß
> Leiter Team IT|Head team IT
> 
> Forum Ziviler Friedensdienst e.V.|Forum Civil Peace Service
> Am Kölner Brett 8 | 50825 Köln | Germany  
> 
> Tel 0221 91273233 | Fax 0221 91273299 | 
> http://www.forumZFD.de 
> 
> Vorstand nach § 26 BGB, einzelvertretungsberechtigt|Executive Board:
> Oliver Knabe (Vorsitz|Chair), Sonja Wiekenberg-Mlalandle, Alexander Mauz  
> VR 17651 Amtsgericht Köln
> 
> Spenden|Donations: IBAN DE37 3702 0500 0008 2401 01 BIC BFSWDE33XXX
> 
> 

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT         
http://catalyst.net.nz/services/samba