[Samba] Replication failures
L.P.H. van Belle
belle at bazuin.nl
Wed May 1 15:16:23 UTC 2019
Hai Mason,
I had a look at the debug output.
on 1) why around 15-16 second, that i really dont know. im trying to figure that out.
on 2) if DNS is inconsistance, that everything is unrelayable.
This is really the first the that needs fixing.
then we look again at the replication.
The debug output still shows several messages about zones in flat files.
I still do believe also that this has impact on your problem.
Your bind config is still not correct, set it exactly as i''ve dont in the howto.
first get everything running error free, then, add you own setting to them.
for example: You ad-dc managed zones still need auth-nxdomain yes;
And i'll have a look at the debug script again since i see it fails at the end.
Im failing to see the big picture here, how your setup is done .. and that does not happen often. :-/
(hint https://www.diagrameditor.com/ )
What i suggest, or what i would do. Verify all needed dns records per server, per host.
I'll sleep a night over this and maybe i can come up with some more.
Your problem is not your samba, but DNS settings and maybe an inheratence from the past..
Greetz,
Louis
Van: M B [mailto:mmx at exm0.net]
Verzonden: woensdag 1 mei 2019 14:44
Aan: L.P.H. van Belle; samba at lists.samba.org
Onderwerp: Re: [Samba] Replication failures
New observations: 1. "samba: task[kccsrv]" always goes to PANIC around 15-16 seconds after samba starts
2. I have three sites and the automatic" NTDS Settings" links between sites are not being generated consistently. I’ve had to manually create some NTDS Seting replication links, especially after I demote/rejoin any DC. I’m guessing the “kccsrv” process should manage these links automatically but it’s crashing so it can not create appropriate links. It seems that links within a site are created automatically, but not necessarily links between sites. I’ve seen links created automatically in some newly re-joined DCs, but not in existing DCs back to the newly re-joined DCs
samba-check-db-repl.sh output pasted below. I pasted results from only one DC. All others are similar. I do get some replication inconsistencies in DNS, but those go away if I run the script again as the differences get resolved
On May 1, 2019, at 2:25 AM, L.P.H. van Belle via samba <samba at lists.samba.org> wrote:
Hai Mason,
-----Oorspronkelijk bericht-----
Van: M B [ MailScanner heeft een e-mail met mogelijk een poging tot fraude gevonden van "exm0.net" mailto:mmx at exm0.net]
Verzonden: dinsdag 30 april 2019 20:42
Aan: L.P.H. van Belle; samba at lists.samba.org
Onderwerp: Re: [Samba] Replication failures
Hi Louis,
In the past few days I’ve removed all bind flat file configs
from my environment, and I’ve checked carefully that all DCs
are replicating and that all changes on any DC eventually
replicate cleanly to all other DCs
Ok, so to confirm, your replication is ok now?
If you think yes, then get en review the setting in this script.
wget https://raw.githubusercontent.com/thctlo/samba4/master/samba-check-db-repl.sh
Run it from every dc and post the outputs.
I’ve checked resolv.conf on all the DCs as well and they all
have at least two other IPs of other DC in them. I believe
you said that the first IP should be the IP of the local
host, but I haven’t done that on every server yet.
Yes, but you change that after the join and after you check replication is ok.
What i always do is, join, reboot, check replication, change dns, reboot, and verify replication again.
This order.
I’m running dc4 on Ubuntu 18.04 using your samba packages.
All other samba DCs are running 4.9.3 that I’ve compiled
previously on Ubuntu 16.04. This same 4.9.3 package is
running without any kcc errors or process PANICs on another
site I manage.
Also, one DC is Windows 2008 R2 (WDC1)
Every time I start samba AD DC on 18.04 with your packages or
on 16.04 with my own packages, the samba kccsvr ( ??????6615
samba: task[kccsrv] ) task starts with all other samba
components and runs for about 10-12 seconds and then goes to
PANIC and crashes as shown in the logs below. After that
‘samba-tool drs showrepl’ always fails.
On the server, set log level = 10
A pain yes, but i dont see directly whats wrong here.
Before a log level 10 post, run on the DC with my packages this again.
https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh
Pm me the unmodified output, i'll re-check that.
What i suspect is a damaged AD or DNS or both.
It's just hard to find, but if AD is replication now, it must be something in the DNS.
I can't tell jet.
I don’t know how to tell if I’m using talloc/tdb from Samba
source or from the OS. I believe it’s from source because I
always compile on a new, clean system and I don’t install any
talloc/tdb or samba packages to prepare the system for compile.
I’ve checked versions as you’ve requested. This version list
is from DC4, with your packages.
ubuntu at dc4:~$ dpkg -l |egrep
"samba|winbin|?db|tevent|talloc|nss|wrapper"
ii dbus 1.12.2-1ubuntu1
amd64 simple interprocess messaging
system (daemon and utilities)
.... Shorted this a bit.
2018.05.09-0ubuntu1~18.04.1 all wireless
regulatory database
This looks ok.
This is from DC5 with my packages. You’ll note that this list
shows "samba-common 2:4.3.11+dfsg-0ubuntu0.16.04.12” but
this is only the folder structure and file structure created
by 4.3.11 Ubuntu package. I found out the hard way that if I
purge that package, it deletes my entire /var/lib/samba
directory, so I had to re-build one of my DC’s from scratch. :(
Au, yes, the other option was to run : apt dist-upgrade
What should have upgraded that package.
Hard, but this way we learn quicker, and.. I know you feeling ;-)
==
ubuntu at dc5:~$ dpkg -l |egrep
"samba|winbin|?db|tevent|talloc|nss|wrapper"
ii dbus 1.10.6-1ubuntu3.3
amd64 simple interprocess
....
2018.05.09-0ubuntu1~16.04.1 all
wireless regulatory database
Here also left overs. In samba packages.
The sources build does include tallec/tevent/tdb/ldb so you dont see these in the list.
And i dont know how you create your samba 4.9.3 package so this is a bit hard to tell.
I suggest,
Stop samba, backup you /var/{lib,cache}/samba/ and /etc/samba
apt remove --purge samba-common samba --autoremove
And install the 4.9.3 back.
Or, upgrade to ubuntu 18.04 and setup my 4.9 repo.
Or use my repo and rebuild the packages for your own.
Greetz,
Louis
Typical output from script:
Running with with console output
Checking the DC_With_FSMO (dc1) with SAMBA DC: dc5.my.company.tld
dc4.my.company.tld
dc7.my.company.tld
dc6.my.company.tld
dc2.my.company.tld
Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc5.my.company.tld
Please wait.. this can take a while..
cat /tmp/samba_ldapcmp_checkdb
* Comparing [DOMAIN] context...
* Objects to be compared: 1321
* Result for [DOMAIN]: SUCCESS
* Comparing [CONFIGURATION] context...
* Objects to be compared: 1713
* Result for [CONFIGURATION]: SUCCESS
* Comparing [SCHEMA] context...
* Objects to be compared: 1550
* Result for [SCHEMA]: SUCCESS
* Comparing [DNSDOMAIN] context...
* Objects to be compared: 1691
* Result for [DNSDOMAIN]: SUCCESS
* Comparing [DNSFOREST] context...
* Objects to be compared: 49
* Result for [DNSFOREST]: SUCCESS
Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc4.my.company.tld
Please wait.. this can take a while..
cat /tmp/samba_ldapcmp_checkdb
* Comparing [DOMAIN] context...
* Objects to be compared: 1321
* Result for [DOMAIN]: SUCCESS
* Comparing [CONFIGURATION] context...
* Objects to be compared: 1713
* Result for [CONFIGURATION]: SUCCESS
* Comparing [SCHEMA] context...
* Objects to be compared: 1550
* Result for [SCHEMA]: SUCCESS
* Comparing [DNSDOMAIN] context...
* Objects to be compared: 1691
* Result for [DNSDOMAIN]: SUCCESS
* Comparing [DNSFOREST] context...
* Objects to be compared: 49
* Result for [DNSFOREST]: SUCCESS
Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc7.my.company.tld
Please wait.. this can take a while..
cat /tmp/samba_ldapcmp_checkdb
* Comparing [DOMAIN] context...
* Objects to be compared: 1321
* Result for [DOMAIN]: SUCCESS
* Comparing [CONFIGURATION] context...
* Objects to be compared: 1713
* Result for [CONFIGURATION]: SUCCESS
* Comparing [SCHEMA] context...
* Objects to be compared: 1550
* Result for [SCHEMA]: SUCCESS
* Comparing [DNSDOMAIN] context...
* Objects to be compared: 1691
* Result for [DNSDOMAIN]: SUCCESS
* Comparing [DNSFOREST] context...
* Objects to be compared: 49
* Result for [DNSFOREST]: SUCCESS
Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc6.my.company.tld
Please wait.. this can take a while..
cat /tmp/samba_ldapcmp_checkdb
* Comparing [DOMAIN] context...
* Objects to be compared: 1321
* Result for [DOMAIN]: SUCCESS
* Comparing [CONFIGURATION] context...
* Objects to be compared: 1714
* Result for [CONFIGURATION]: SUCCESS
* Comparing [SCHEMA] context...
* Objects to be compared: 1550
* Result for [SCHEMA]: SUCCESS
* Comparing [DNSDOMAIN] context...
* Objects to be compared: 1691
* Result for [DNSDOMAIN]: SUCCESS
* Comparing [DNSFOREST] context...
* Objects to be compared: 49
* Result for [DNSFOREST]: SUCCESS
Running : /usr/bin/samba-tool ldapcmp --filter="whenChanged,dc,DC,cn,CN" ldap://dc1.my.company.tld ldap://dc2.my.company.tld
Please wait.. this can take a while..
cat /tmp/samba_ldapcmp_checkdb
* Comparing [DOMAIN] context...
* Objects to be compared: 1321
* Result for [DOMAIN]: SUCCESS
* Comparing [CONFIGURATION] context...
* Objects to be compared: 1714
* Result for [CONFIGURATION]: SUCCESS
* Comparing [SCHEMA] context...
* Objects to be compared: 1550
* Result for [SCHEMA]: SUCCESS
* Comparing [DNSDOMAIN] context...
* Objects to be compared: 1691
* Result for [DNSDOMAIN]: SUCCESS
* Comparing [DNSFOREST] context...
* Objects to be compared: 49
* Result for [DNSFOREST]: SUCCESS
.. Next check..
Running : samba-tool drs showrepl
grep -c "failed" /tmp/samba_drs_showrepl
grep -c "successful" /tmp/samba_drs_showrepl
failures don't match
successes don't match
failures don't match
successes don't match
failures don't match
successes don't match
failures don't match
successes don't match
failures don't match
successes don't match
if [ "${EMAIL_REPORT_ALWAYS}" = "yes" ] && [ -n "${EMAIL_REPORT_ADDRESS}" ]; then
#cat /tmp/samba_drs_showrepl | ${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS
${SET_MAILTOOL} -s "SAMBA CHECK DB : showrepl results" $EMAIL_REPORT_ADDRESS < /tmp/samba_drs_showrepl
#cat /tmp/samba_ldapcmp_checkdb | ${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS
${SET_MAILTOOL} -s "SAMBA CHECK DB : ldapcmp results" $EMAIL_REPORT_ADDRESS < /tmp/samba_ldapcmp_checkdb
fi
if [ "${SETREMOVELOG}" = "yes" ]; then
if [ -f /tmp/samba_ldapcmp_checkdb ]; then
rm /tmp/samba_ldapcmp_checkdb
fi
if [ -f /tmp/samba_drs_showrepl ]; then
rm /tmp/samba_drs_showrepl
fi
fi
More information about the samba
mailing list