[Samba] (SOLVED) NT_STATUS_INTERNAL_DB_CORRUPTION messages in log.samba--proper course of action?

Tue Jul 7 13:23:08 UTC 2015

(Sorry for the other list users that my previous reply to this thread
went to Rowland Penny only. That wasn't my intention.)

On 03/07/15 21:07, Rowland Penny wrote:
> On 03/07/15 18:51, Pinja-Liina Jalkanen wrote:
>> On 03/07/15 17:32, Rowland Penny wrote:
>>> On 03/07/15 14:25, Pinja-Liina Jalkanen wrote:
>>>> Hi all,
>>>>
>>>> We've recently migrated from a separate DNS server that was dynamically
>>>> updated with BIND's update-policy, using a manually generated
>>>> tkey-gssapi-keytab (plus a second server functioning as an ordinary
>>>> slave to the first), to BIND9_DLZ. The setup predated Samba's AD DC
>>>> support and BIND's DLZ support, and was originally established because
>>>> even though we needed AD, we were unwilling to use Windows's own DNS
>>>> server.
> 
> OK, so I missed that you hadn't provisioned samba4, but the message when
> you run 'samba-tool domain join --help' is even more explicit :
> 
>    --dns-backend=NAMESERVER-BACKEND
>                         The DNS server backend. SAMBA_INTERNAL is the
> builtin
>                         name server (default), BIND9_DLZ uses samba4 AD to
>                         store zone information, NONE skips the DNS setup
>                         entirely (this DC will not be a DNS server)
> 
> You need DNS for an AD domain, no ifs or buts,

Very true. And we've always had one.

> and experience of this
> mailing list leads to me think that not running it on the DCs is a bad
> idea.

That's entirely by preference. The only thing that you can't achieve
with separate DNS servers is multiple dynamically updatable servers,
because with separate servers only the one acting as a primary can be
dynamically updated. Secure dynamic updates have been possible from BIND
9.5 forwards, if I recall correctly. We first implemented them with BIND
9.7.

>>> Why did you go with '--dns-backend=None' , did you miss the 'NONE skips
>>> the  DNS setup entirely (not recommended)' part in the commands help?
>>> Don't bother answering, this is a rhetorical question.
>> You're throwing me rethorical questions, but didn't bother to actually
>> _read_ my message, did you? I explained quite carefully that we used to
>> have a DNS setup that is separate from AD and that _predates_ Samba's AD
>> support--that is Samba 4.0--entirely.
>>
>> You could have just as well asked me why we didn't back then just go
>> with the MS DNS but decided to use BIND instead. Because we've never
>> ever _provisioned_ Samba; at least not as in "samba-tool domain
>> provision". The "provisioning" of our domain was, once upon a time, done
>> with Windows' dcpromo.exe.
>>
>> When we first added a Samba DC to the mix we were absolutely NOT going
>> to change the existing DNS setup, as Samba's AD support was still very
>> bleeding edge and that's why the first Samba DC was joined to our domain
>> with --dns-backend=NONE. This whole problem arose when we were finally
>> brave enough to try to change that setup, but there wasn't any
>> documentation explaining how to do that.
> 
> There isn't any documentation for what you need to do now, because
> nobody ever thought that somebody would set up AD with samba4 (in any
> form) without a DNS server running on the DC.

Except the person who added the "NONE" option to the code, perhaps?

And Microsoft certainly had thought about it--I vaguely recall them
having documentation how to manually register the records required by
the DC back in 2000 when AD was first launched. Migrating the DNS to
Windows was never mandatory, and many were never thrilled to run MS DNS.
Like we weren't. But for some of us, Samba NT domain wasn't enough, either.

> Your only hope is to go through the files in
> /usr/share/pyshared/samba/provision/ and pick out the required info from
> them.

Actually, no. It wasn't my only hope. My original mistake was that I
should have demoted all non-FSMO DCs first and not join any others
before upgrading, but run samba_upgradedns --dns_backend=BIND9_DLZ on
that remaining single DC. Then join further DCs again, this time with
the right DNS backend. I'm pretty certain now that this would've worked
right away. The INTERNAL_DB_CORRUPTION message was just due to the old
DC not being aware of all the changes that had only been made on the
newly joined DC.

So what did I do to actually fix our setup? (DCs A, B and C refer to my
original post in this thread.)

1. Made sure that I had good backups of everything.
2. Transferred FSMO roles from DC B to DC A. No errors.
3. Demoted DC B, so as to have only one DC (A) left. No errors. Then ran
metadata cleanup just to make it sure.
4. Joined DC C, with option --dns_backend=BIND9_DLZ. No errors.
5. Performed the "Check and fix DNS entries on DC joins" procedure (see
https://wiki.samba.org/index.php/Check_and_fix_DNS_entries_on_DC_joins).
6. Hey presto, it works! No errors left anywhere anymore!

(Actually it didn't quite go this easily, as I accidentally pointed the
DC C's objectGUID CNAME record to DC A first, which totally broke
replication after joining of the DC C. But after spotting the error it
was easy to fix!)

> I certainly wont be helping you any further in this problem of
> your own making!

Your comment irks me rather badly. We had very good reasons (unless
you're a great fan of MS) for our previous setup, and the "NONE" option
for the DNS backend was always there. I think it's downright vile from
your part to just label it "our own making"--like if we'd hacked the
Samba source with our private patches or the like and were now asking
for help. If you don't want to help someone, I suggest you just keep
quiet instead!

And if you really think the "NONE" option shouldn't be there at all, I
think you can file a bug to remove it.

-----------

For others potentially being in the same situation:

1. Make good backups!
2. Make a script that calls "samba-tool dns add" to migrate your old
zonefile(s).
3. Strip your network down to a single DC (remember FSMO transfers and
metadata cleanups!).
4. Create the DnsAdmins security group.
5. Remove any conflicting SPNs, if any.
6. Shutdown Samba.
7. Upgrade your domain using "samba_upgradedns --dns-backend=BIND9_DLZ".
8. Setup BIND (see Samba wiki) on your DC and start it.
9. Point your DC to resolve DNS from localhost, if it isn't already. 10.
Start Samba and run "samba_dnsupdate --all-names --verbose
--fail-immediately"; it should pass.
11. Migrate your old zone(s) using the script you made at #2.
12. When you've confirmed that everything works, you can join other DCs
again.

>> By the way, while the DNS updates do work now, I happened to notice the
>> following message in the BIND log right after a successful update. It's
>> most likely related:
>>
>> Jul  3 19:00:42 dc-a named[18846]: failed to find dnsRecord for
>> DC=mydomain.tld,CN=MicrosoftDNS,DC=DomainDnsZones,DC=mydomain,DC=tld

OK, this seems nothing to be worried about; named complains about this
every time a Samba managed zone is AXFR'd (e.g. if one runs "dig @dc-a
_msdcs.mydomain.tld. AXFR").

-- 
Pinja-Liina Jalkanen
Vihreät / De Gröna
https://www.vihreat.fi/