[Samba] replication fails after internal error 11 / panic
lists
lists at merit.unu.edu
Tue Jan 5 10:09:37 UTC 2016
Hi,
We have three DC's, and one of them has been misbehaving a few times
lately, stopping to replicate, showing the following error in samba-tool
drs showrepl, for all DC partitions:
> DC=DomainDnsZones,DC=samba,DC=company,DC=com
> Default-First-Site-Name\DC2 via RPC
> DSA object GUID: 5e93a102-2963-496a-af16-0c51eebb2e31
> Last attempt @ Wed Nov 11 06:41:21 2015 CET failed, result 121 (WERR_SEM_TIMEOUT)
> 4 consecutive failure(s).
> Last success @ Wed Nov 11 06:24:34 2015 CET
Similar behaviour for this DC2 has also been observed when this DC2 was
still on 4.1.17-sernet. I upgraded to 4.2.5-sernet, hoping that this
would perhaps solve the issue.
I have a loglevel 3 log from that DC2, and it shows the following:
> [2015/11/11 06:25:10.644343, 0] ../lib/util/fault.c:78(fault_report)
> ===============================================================
> [2015/11/11 06:25:10.665628, 0] ../lib/util/fault.c:79(fault_report)
> INTERNAL ERROR: Signal 11 in pid 5871 (4.2.5-SerNet-Debian-8.wheezy)
> Please read the Trouble-Shooting section of the Samba HOWTO
> [2015/11/11 06:25:10.665969, 0] ../lib/util/fault.c:81(fault_report)
> ===============================================================
> [2015/11/11 06:25:10.666235, 0] ../lib/util/fault.c:151(smb_panic_default)
> PANIC: internal error
> [2015/11/11 06:25:11.944174, 0] ../source4/smbd/server.c:370(binary_smbd_main)
> samba version 4.2.5-SerNet-Debian-8.wheezy started.
This DC2 has only been running very short, after a scheduled reboot for
backup:
> [2015/11/11 06:25:07.434308, 0] ../source4/smbd/server.c:370(binary_smbd_main)
> samba version 4.2.5-SerNet-Debian-8.wheezy started.
> Copyright Andrew Tridgell and the Samba Team 1992-2014
...cutting many uninteresting "GENSEC/AUTH/NTVFS backend registered"
lines, and then samba suddenly seems to start AGAIN..:
> [2015/11/11 06:25:07.682194, 3] ../source4/param/share.c:124(share_register)
> SHARE backend [ldb] registered.
> [2015/11/11 06:25:07.710416, 0] ../source4/smbd/server.c:116(sig_term)
> SIGTERM: killing children
> [2015/11/11 06:25:07.711006, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5802 on SIGTERM
> [2015/11/11 06:25:07.871830, 0] ../source4/smbd/server.c:370(binary_smbd_main)
> samba version 4.2.5-SerNet-Debian-8.wheezy started.
> Copyright Andrew Tridgell and the Samba Team 1992-2014
> [2015/11/11 06:25:07.872809, 3] ../source4/smbd/server.c:381(binary_smbd_main)
> Becoming a daemon.
> [2015/11/11 06:25:07.893288, 3] ../auth/gensec/gensec_start.c:891(gensec_register)
> GENSEC backend 'gssapi_spnego' registered
...cutting again many uninteresting "GENSEC/AUTH/NTVFS backend
registered" lines
and then samba seems to start AGAIN, generating the INTERNAL ERROR / PANIC:
> [2015/11/11 06:25:09.195430, 2] ../source4/dsdb/kcc/kcc_service.c:127(kccsrv_load_partitions)
> kccsrv_partition[DC=ForestDnsZones,DC=samba,DC=company,DC=com] loaded
> [2015/11/11 06:25:09.220257, 0] ../source4/smbd/server.c:116(sig_term)
> SIGTERM: killing children
> [2015/11/11 06:25:09.220557, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5839 on SIGTERM
> [2015/11/11 06:25:09.222634, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5865 on SIGTERM
> [2015/11/11 06:25:09.226357, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5863 on SIGTERM
> [2015/11/11 06:25:09.229021, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5858 on SIGTERM
> [2015/11/11 06:25:09.230342, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5857 on SIGTERM
> [2015/11/11 06:25:09.234441, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5851 on SIGTERM
> [2015/11/11 06:25:09.238290, 0] ../source4/smbd/server.c:121(sig_term)
> [2015/11/11 06:25:09.239373, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5845 on SIGTERM
> Exiting pid 5849 on SIGTERM
> [2015/11/11 06:25:09.239002, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5846 on SIGTERM
> [2015/11/11 06:25:09.250624, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5867 on SIGTERM
> [2015/11/11 06:25:09.294308, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5853 on SIGTERM
> [2015/11/11 06:25:09.310302, 0] ../source4/smbd/server.c:121(sig_term)
> Exiting pid 5871 on SIGTERM
> [2015/11/11 06:25:10.644343, 0] ../lib/util/fault.c:78(fault_report)
> ===============================================================
> [2015/11/11 06:25:10.665628, 0] ../lib/util/fault.c:79(fault_report)
> INTERNAL ERROR: Signal 11 in pid 5871 (4.2.5-SerNet-Debian-8.wheezy)
> Please read the Trouble-Shooting section of the Samba HOWTO
> [2015/11/11 06:25:10.665969, 0] ../lib/util/fault.c:81(fault_report)
> ===============================================================
> [2015/11/11 06:25:10.666235, 0] ../lib/util/fault.c:151(smb_panic_default)
> PANIC: internal error
> [2015/11/11 06:25:11.944174, 0] ../source4/smbd/server.c:370(binary_smbd_main)
> samba version 4.2.5-SerNet-Debian-8.wheezy started.
> Copyright Andrew Tridgell and the Samba Team 1992-2014
... again cutting many uninteresting "GENSEC/AUTH/NTVFS backend
registered" lines, but then errors start appearing:
> [2015/11/11 06:25:13.279170, 2] ../source4/dsdb/kcc/kcc_service.c:127(kccsrv_load_partitions)
> kccsrv_partition[DC=samba,DC=company,DC=com] loaded
> [2015/11/11 06:25:13.279312, 2] ../source4/dsdb/kcc/kcc_service.c:127(kccsrv_load_partitions)
> kccsrv_partition[DC=DomainDnsZones,DC=samba,DC=company,DC=com] loaded
> [2015/11/11 06:25:13.279465, 2] ../source4/dsdb/kcc/kcc_service.c:127(kccsrv_load_partitions)
> kccsrv_partition[DC=ForestDnsZones,DC=samba,DC=company,DC=com] loaded
> [2015/11/11 06:25:13.293693, 3] ../source4/dsdb/dns/dns_update.c:340(dnsupdate_check_names)
> Calling DNS name update script
> [2015/11/11 06:25:13.343947, 3] ../lib/ldb-samba/ldb_wrap.c:321(ldb_wrap_connect)
> ldb_wrap open of secrets.ldb
> [2015/11/11 06:25:13.358502, 3] ../source4/dsdb/dns/dns_update.c:355(dnsupdate_check_names)
> Calling SPN name update script
> [2015/11/11 06:25:14.546779, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
> Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED'
> [2015/11/11 06:25:14.552054, 3] ../source4/smbd/process_single.c:114(single_terminate)
> single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]
> [2015/11/11 06:25:14.585066, 3] ../lib/ldb-samba/ldb_wrap.c:321(ldb_wrap_connect)
> ldb_wrap open of secrets.ldb
> [2015/11/11 06:25:14.918680, 3] ../source4/rpc_server/drsuapi/dcesrv_drsuapi.c:74(dcesrv_drsuapi_DsBind)
> ../source4/rpc_server/drsuapi/dcesrv_drsuapi.c:74: doing DsBind with system_session
> [2015/11/11 06:25:14.953473, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (c94b2499-9cc1-4006-9c6e-15f0b45195c0, DC=DomainDnsZones,DC=samba,DC=company,DC=com, 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com)
> [2015/11/11 06:25:14.969835, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (4105c810-4199-4656-b8bf-7e5cdbc806cc, DC=ForestDnsZones,DC=samba,DC=company,DC=com, 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com)
> [2015/11/11 06:25:14.983136, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (a8bce1ff-7069-4a19-a431-10b1030a091c, CN=Configuration,DC=samba,DC=company,DC=com, 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com)
> [2015/11/11 06:25:14.995523, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (889bb0b6-78d7-4308-8d87-3cd1fadcd0ac, DC=samba,DC=company,DC=com, 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com)
> [2015/11/11 06:25:15.015289, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (202e8a1f-e148-45cf-9529-a3dccd6bfbf4, CN=Schema,CN=Configuration,DC=samba,DC=company,DC=com, 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com)
> [2015/11/11 06:25:15.921925, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
> Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED'
> [2015/11/11 06:25:15.922318, 3] ../source4/smbd/process_single.c:114(single_terminate)
> single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]
> [2015/11/11 06:25:15.959523, 3] ../lib/ldb-samba/ldb_wrap.c:321(ldb_wrap_connect)
> ldb_wrap open of secrets.ldb
> [2015/11/11 06:25:15.963878, 3] ../libcli/nbt/lmhosts.c:185(resolve_lmhosts_file_as_sockaddr)
> resolve_lmhosts: Attempting lmhosts lookup for name 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com<0x20>
> [2015/11/11 06:25:16.024192, 3] ../source4/rpc_server/drsuapi/dcesrv_drsuapi.c:74(dcesrv_drsuapi_DsBind)
> ../source4/rpc_server/drsuapi/dcesrv_drsuapi.c:74: doing DsBind with system_session
> [2015/11/11 06:25:16.055259, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (c94b2499-9cc1-4006-9c6e-15f0b45195c0, DC=DomainDnsZones,DC=samba,DC=company,DC=com, 59f2b0eb-9e21-437d-a6d0-178edabee2b3._msdcs.samba.company.com)
> [2015/11/11 06:25:16.074855, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (4105c810-4199-4656-b8bf-7e5cdbc806cc, DC=ForestDnsZones,DC=samba,DC=company,DC=com, 59f2b0eb-9e21-437d-a6d0-178edabee2b3._msdcs.samba.company.com)
> [2015/11/11 06:25:16.086656, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (a8bce1ff-7069-4a19-a431-10b1030a091c, CN=Configuration,DC=samba,DC=company,DC=com, 59f2b0eb-9e21-437d-a6d0-178edabee2b3._msdcs.samba.company.com)
> [2015/11/11 06:25:16.093633, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (889bb0b6-78d7-4308-8d87-3cd1fadcd0ac, DC=samba,DC=company,DC=com, 59f2b0eb-9e21-437d-a6d0-178edabee2b3._msdcs.samba.company.com)
> [2015/11/11 06:25:16.103275, 3] ../source4/dsdb/repl/drepl_service.c:202(_drepl_schedule_replication)
> _drepl_schedule_replication: forcing sync of partition (202e8a1f-e148-45cf-9529-a3dccd6bfbf4, CN=Schema,CN=Configuration,DC=samba,DC=company,DC=com, 59f2b0eb-9e21-437d-a6d0-178edabee2b3._msdcs.samba.company.com)
> [2015/11/11 06:25:16.184653, 3] ../libcli/nbt/lmhosts.c:185(resolve_lmhosts_file_as_sockaddr)
> resolve_lmhosts: Attempting lmhosts lookup for name 9a3d9130-45f3-43b6-bbf4-189c19764bd5._msdcs.samba.company.com<0x20>
> [2015/11/11 06:25:16.371331, 3] ../lib/ldb-samba/ldb_wrap.c:321(ldb_wrap_connect)
> ldb_wrap open of secrets.ldb
> [2015/11/11 06:25:16.569542, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
> Kerberos: TGS-REQ DC2$@samba.company.com from ipv4:1.2.3.15:42187 for GC/DC3.samba.company.com/samba.company.com at samba.company.com [canonicalize]
> [2015/11/11 06:25:16.695476, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
> Kerberos: TGS-REQ authtime: 2015-11-11T06:25:16 starttime: 2015-11-11T06:25:16 endtime: 2015-11-11T16:25:16 renew till: unset
> [2015/11/11 06:25:16.827388, 2] ../source4/dsdb/repl/replicated_objects.c:944(dsdb_replicated_objects_commit)
> Replicated 0 objects (0 linked attributes) for DC=DomainDnsZones,DC=samba,DC=company,DC=com
> [2015/11/11 06:25:16.847477, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered DC2<00> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.847700, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered DC2<03> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.847736, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered DC2<20> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.911943, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered SAMDOM<1b> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.912043, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered SAMDOM<1c> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.912079, 3] ../source4/nbt_server/register.c:154(nbtd_register_name_handler)
> Registered SAMDOM<00> with 1.2.3.15 on interface 1.2.3.255
> [2015/11/11 06:25:16.946674, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
> Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED'
> [2015/11/11 06:25:16.947050, 3] ../source4/smbd/process_single.c:114(single_terminate)
> single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]
> [2015/11/11 06:25:16.968982, 2] ../source4/dsdb/repl/replicated_objects.c:944(dsdb_replicated_objects_commit)
> Replicated 0 objects (0 linked attributes) for DC=ForestDnsZones,DC=samba,DC=company,DC=com
> [2015/11/11 06:25:16.990366, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
> Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED'
> [2015/11/11 06:25:16.990714, 3] ../source4/smbd/process_single.c:114(single_terminate)
> single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]
> [2015/11/11 06:25:17.026260, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
> Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED'
> [2015/11/11 06:25:17.026644, 3] ../source4/smbd/process_single.c:114(single_terminate)
> single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]
Ideas, anyone?
MJ
More information about the samba
mailing list