[Samba] After upgrade to 4.9.4, internal DNS no longer working
Viktor Trojanovic
viktor at troja.ch
Wed Dec 26 10:43:37 UTC 2018
I could really use some support with this. I understand it's always
possible to just restore from a backup but the more interesting question is
if something can be done with the data at hand. Basically, I'm trying to
understand how it's possible that a dbcheck shows no errors, an ldbsearch
is successful, and yet it's not possible to start the AD properly. What
else is there that could be corrupted, and is there a way to repair it?
Thanks.
On Sun, 23 Dec 2018 at 13:12, Viktor Trojanovic <viktor at troja.ch> wrote:
> The two systems I'm referring to are on different domains, different
> locations. No connection whatsoever except me being the one administering
> both.
>
> According to oEMInformation the server was provisioned by 4.3.3. I just
> checked the Samba log file and it seems that version 4.7.4 was running
> before the update.
>
> I'm now looking at the logs right before the upgrade and indeed, there
> have been errors so it looks like we found the reason for my issues. As a
> matter of fact, just a day before the upgrade, we had an out of memory
> error on the server and got into the situation that we had to make a hard
> reset. As I can see now, Samba didn't go through this chain of events
> unharmed. It was careless of me not to check but since the clients seemed
> to be able to talk to the AD, I just didn't think there is an issue with
> Samba.
>
> Here are the respective log entries. In the beginning, they refer to
> version 4.7.4.:
>
> [... prior log entry from a month earlier, no issues. Errors and warning
> start after out of memory issue on the server ...]
>
> [2018/12/21 03:06:02.974778, 0]
> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
> ../source4/dsdb/dns/dns_update.c:290: Failed DNS update - with error
> code 110
> [2018/12/21 03:16:03.413186, 0]
> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
> ../source4/dsdb/dns/dns_update.c:290: Failed DNS update - with error
> code 110
> [2018/12/21 03:16:03.741426, 0]
> ../source4/dsdb/dns/dns_update.c:313(dnsupdate_spnupdate_done)
> ../source4/dsdb/dns/dns_update.c:313: Failed SPN update - with error
> code 110
> [2018/12/21 03:26:03.562027, 0]
> ../source4/dsdb/dns/dns_update.c:290(dnsupdate_nameupdate_done)
> ../source4/dsdb/dns/dns_update.c:290: Failed DNS update - with error
> code 110
> [2018/12/21 03:26:03.831110, 0]
> ../source4/dsdb/dns/dns_update.c:313(dnsupdate_spnupdate_done)
> ../source4/dsdb/dns/dns_update.c:313: Failed SPN update - with error
> code 110
> [2018/12/21 03:27:58.903344, 0]
> ../source4/dsdb/kcc/kcc_periodic.c:693(samba_kcc_done)
> ../source4/dsdb/kcc/kcc_periodic.c:693: Failed samba_kcc -
> NT_STATUS_IO_TIMEOUT
> [2018/12/21 03:32:59.630451, 0]
> ../source4/dsdb/kcc/kcc_periodic.c:693(samba_kcc_done)
> ../source4/dsdb/kcc/kcc_periodic.c:693: Failed samba_kcc -
> NT_STATUS_IO_TIMEOUT
>
> [... same errors repeat about 100 times, deleted for better legibility
> ...]
>
> [2018/12/21 09:17:59.389008, 0]
> ../source4/dsdb/dns/dns_update.c:313(dnsupdate_spnupdate_done)
> ../source4/dsdb/dns/dns_update.c:313: Failed SPN update - with error
> code 12
> [2018/12/21 09:19:38.518542, 0]
> ../source4/dsdb/kcc/kcc_periodic.c:693(samba_kcc_done)
> ../source4/dsdb/kcc/kcc_periodic.c:693: Failed samba_kcc -
> NT_STATUS_NO_MEMORY
> [2018/12/21 09:22:52.899142, 0]
> ../source4/smbd/server.c:450(binary_smbd_main)
> samba version 4.7.4 started.
> Copyright Andrew Tridgell and the Samba Team 1992-2017
> [2018/12/21 09:22:55.544887, 0]
> ../source4/smbd/server.c:622(binary_smbd_main)
> samba: using 'standard' process model
> [2018/12/21 09:22:55.565451, 0]
> ../lib/util/become_daemon.c:124(daemon_ready)
> STATUS=daemon 'samba' finished starting up and ready to serve connections
> samba: setproctitle not initialized, please either call
> setproctitle_init() or link against libbsd-ctor.
>
> [... warning above repeats about 10 times ...]
>
> [2018/12/22 19:13:44.916007, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: Traceback (most recent call last):
> [2018/12/22 19:13:44.925489, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: File "/usr/bin/samba_kcc", line 45, in <module>
> [2018/12/22 19:13:44.925556, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: from samba import getopt as options
> [2018/12/22 19:13:44.925595, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: File
> "/usr/lib/python2.7/site-packages/samba/__init__.py", line 29, in <module>
> [2018/12/22 19:13:44.999014, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: import samba.param
> [2018/12/22 19:13:44.999099, 0]
> ../lib/util/util_runcmd.c:327(samba_runcmd_io_handler)
> /usr/bin/samba_kcc: ImportError: /usr/lib/samba/libreplace-samba4.so:
> version `SAMBA_4.7.4' not found (required by /usr/lib/libsamba-util.so.0)
> [2018/12/22 19:13:45.004497, 0]
> ../source4/dsdb/kcc/kcc_periodic.c:693(samba_kcc_done)
> ../source4/dsdb/kcc/kcc_periodic.c:693: Failed samba_kcc -
> NT_STATUS_ACCESS_DENIED
> [2018/12/22 19:31:48.283948, 0]
> ../source4/smbd/process_standard.c:86(sigterm_signal_handler)
> Exiting pid 391 on SIGTERM
>
> [... error above repeats about 10 times ...]
>
> [2018/12/22 19:31:48.446449, 0] ../source4/smbd/server.c:131(sig_term)
> SIGTERM: killing children
> [2018/12/22 19:31:48.447387, 0] ../source4/smbd/server.c:131(sig_term)
> SIGTERM: killing children
> [2018/12/22 19:31:48.447428, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 367 on SIGTERM
> [2018/12/22 19:31:48.464044, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 391 on SIGTERM
> [2018/12/22 19:31:48.806419, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 394 on SIGTERM
> [2018/12/22 19:31:48.806490, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 395 on SIGTERM
> [2018/12/22 19:31:48.806540, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 399 on SIGTERM
> [2018/12/22 19:31:48.806590, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 392 on SIGTERM
> [2018/12/22 19:31:48.806634, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 458 on SIGTERM
> [2018/12/22 19:31:48.806679, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 393 on SIGTERM
> [2018/12/22 19:31:48.811314, 0] ../source4/smbd/server.c:135(sig_term)
> [2018/12/22 19:31:48.811348, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 401 on SIGTERM
> Exiting pid 396 on SIGTERM
> [2018/12/22 19:31:48.811404, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 400 on SIGTERM
> [2018/12/22 19:31:48.811451, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 402 on SIGTERM
> [2018/12/22 19:31:48.811511, 0] ../source4/smbd/server.c:135(sig_term)
> [2018/12/22 19:31:48.811546, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 398 on SIGTERM
> [2018/12/22 19:31:48.811605, 0] ../source4/smbd/server.c:135(sig_term)
> Exiting pid 390 on SIGTERM
> Exiting pid 397 on SIGTERM
> [2018/12/22 19:35:38.400284, 0]
> ../source4/smbd/server.c:510(binary_smbd_main)
>
> [... clearly, I made an upgrade at this point, not aware I had problems.
> what follows are issues I already described ...]
>
> samba version 4.9.4 started.
> Copyright Andrew Tridgell and the Samba Team 1992-2018
> [2018/12/22 19:35:40.471526, 0]
> ../source4/smbd/server.c:696(binary_smbd_main)
> binary_smbd_main: samba: using 'standard' process model
> [2018/12/22 19:35:40.647282, 0]
> ../source4/smbd/service_task.c:36(task_server_terminate)
> [2018/12/22 19:35:40.647380, 0]
> ../source4/dsdb/common/util.c:1815(samdb_reference_dn_is_our_ntdsa)
> Failed to find object DC=samdom,DC=example,DC=com for attribute
> fsmoRoleOwner - Cannot find DN DC=samdom,DC=example,DC=com to get attribute
> fsmoRoleOwner for reference dn: No such Base DN: DC=samdom,DC=example,DC=com
> [2018/12/22 19:35:40.648352, 0]
> ../source4/dsdb/dns/dns_update.c:127(dnsupdate_rebuild)
> task_server_terminate: task_server_terminate: [kdc: krb5_init_context
> samdb RODC connect failed]
> [2018/12/22 19:35:40.674777, 0]
> ../source4/smbd/service_task.c:36(task_server_terminate)
> [2018/12/22 19:35:40.674823, 0]
> ../source4/smbd/service_task.c:36(task_server_terminate)
> task_server_terminate: task_server_terminate: [dreplsrv: Failed to
> connect to local samdb: WERR_DS_UNAVAILABLE
>
>
> It looks like the memory issue corrupted something in Samba. But what and
> why? I'm sure I didn't make it better by upgrading, I assume this
> reindexing you referred to didn't really happen if Samba was already in a
> degraded state. On the other hand, if the ldb-files can be queried
> properly, where is the corruption? When it comes to these details, Samba is
> still pretty much a black box to me.
>
> Thanks a lot for bearing with me so far. I hope this can be solved
> somehow. I guess the good thing is that, if worse comes to worst, I could
> just downgrade back to 4.7. and restore a backup from before the memory
> issue. Would it be alright, in that worst case, if I just overwrite all
> contents of /var/lib/samba with the files I have in the backup?
>
>
> On Sun, 23 Dec 2018 at 11:36, Rowland Penny via samba <
> samba at lists.samba.org> wrote:
>
>> On Sun, 23 Dec 2018 11:19:37 +0100
>> Viktor Trojanovic <viktor at troja.ch> wrote:
>>
>> > I was too quick to send my previous email. It can't really be a
>> > problem of the package since I have, as mentioned, another system
>> > with the exact same setup and the exact same package versions and it
>> > all works there...
>> >
>>
>> Upgrading a running Samba server on an OS should work without error,
>> unless there was an underlying problem already.
>>
>> Is your other system running as a DC in the same domain, or is it a
>> totally separate domain ?
>>
>> It might help if you could remember what version you upgraded from, was
>> it the version it was provisioned at ?
>> If so, there is an attribute in the base 'dc=samdom,dc=example,dc',
>> this attribute 'oEMInformation' contains the Samba version at provision.
>>
>> Rowland
>>
>> --
>> To unsubscribe from this list go to the following URL and read the
>> instructions: https://lists.samba.org/mailman/options/samba
>
>
More information about the samba
mailing list