More KCC patches

Thu Jul 9 22:09:47 UTC 2015

G'Day Douglas and Andrew,

>>> I believe so, and thank you for trying!
>>
>> You are welcome. I am eager to see it working: I have a client who is
>> deploying Samba4 on a primary school network and currently have more
>> than 55 DCs on as many sites in one domain. I am quite wary about the
>> current full meshed replication topology that the current KCC is doing,
>> and I am eager to help getting samba_kcc to work :-)
>
> I did a new checkout of kcc-intersite-29 branch and found time today to
> look deeper into the issue. Actually I realized that I had no bridgehead
> defined for IP transport on the second site. Former kcc didn't care, but
> the new one take that into account.
>
> So now in a two site real life scenario, it gives the same results as
> the old kcc. I'll try tomorrow on a three sites domains.

I haven't encountered any issue that would have required to roll back to 
standard 4.2.2. I am testing with :Version 4.3.0pre1-GIT-c7cbf0c with 
the line kccsrv:samba_kcc=true in smb.conf

* 1-DC domain x3 : no regression (three differents domain on three 
sites), Yeah, no real need for KCC, but still...

* 2-DCs domain x2 : no regression (two differents domains, each with two 
sites). samba_kcc is more stringeant on the configuration, and seems not 
to create the ntdsconnection if there is no bridgehead server defined. 
Other than that it works fine.

* 1-RWDC + 2-RODC domain : it didn't work that well (but former kcc is 
not doing well in this scenario either). Running samba_kcc on the RWDC 
did build a inbound connexion from one of the RODC in the ADS&S console 
(but not the other one...) and didn't create the outbound connexions to 
the RODCs.

However "samba-tool drs showrepl" does not show any inbound connexion 
from RODC (is it only using the repsfrom repsto attributes?). I have 
seen other strange stuff to cleanup on that domain, so don't worry too 
much about that spurious inbound connexion.

On the RODC, samba_kcc crashes :
[root at srvads-retz.mache ~]# samba_kcc
Traceback (most recent call last):
   File "/usr/local/samba/sbin/samba_kcc", line 311, in <module>
     attempt_live_connections=opts.attempt_live_connections)
   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 2534, in run
     self.remove_unneeded_ntdsconn(all_connected)
   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 589, in remove_unneeded_ntdsconn
     self._mark_unneeded_intersite_ntdsconn()
   File 
"/usr/local/samba/lib/python2.7/site-packages/samba/kcc/__init__.py", 
line 549, in _mark_unneeded_intersite_ntdsconn
     for site in self.sites_table.values():
AttributeError: 'KCC' object has no attribute 'sites_table'

After deleting the inbound ntdsconnexion on the two RODCs, they were not 
automagically rebuild and I had to create them by hand. Replication is 
back to normal afterward.

Tomorow I'll continue validating the new samba_kcc on a 3-RWDC domain.

Is it possible to have the samba_kcc modification cherrypicked as a 
patchset that could be applied on a 4.2.2 tree? It will make it easier 
for me to have candidates with larger network for validation.

By the way, is it necessary to updates all the DCs in the domain to run 
samba_kcc, especially with --readonly and --export-ldif or 
--dot-file-dir options. Is it even necessary to join the DC to the 
domain if one uses the -H ldap:// option?

Cheers,

Denis

>
> Cheers,
>
> Denis
>
>>
>>>> I checked out the kcc-intersite-29 branch on a 2 samba DCs replicating
>>>> the same domain (upgraded from 4.2.0):
>> ...
>>>> The test domain has seen almost all the samba releases from the late
>>>> beta onward (and had quite a tumultuous life), so perhaps I'll try
>>>> tomorow on a more pristine domain...
>>>
>>> Well, real life Samba installs have tumultuous lives too, so this is
>>> still a useful test.
>>>
>>> Would it be possible for us to get an ldif dump of the network? Via
>>> something like this:
>>>
>>> ./bin/samba_kcc --exportldif=mysite.ldif -H
>>> ldap://win2012r2-2.ad.example.com \
>>>      -Uadministrator%password
>>>
>>> Please send it privately if there's anything secret about the network.
>>  >
>>>> Another thing : if I delete all the automatically created connexion and
>>>> then create the connexion by hand, replication is working fine,
>>>> however,
>>>> when running samba_kcc, I then get the following error :
>>>>
>>>> [root at srvads.tranq samba]# samba_kcc
>>>> 'module' object has no attribute 'replSchedule'
>>>>
>>>> Taking a look at the drsblobs object :
>>>>
>>>>>>> drsblobs.replSchedule
>> ...
>>> One thing that has caught us out a few times is the *.pyc files that
>>> python compiles from the .py modules. The build process doesn't track
>>> them, and python works out when to regenerate using the timestamps. A
>>> git checkout gives files the timestamps that git thinks they should,
>>> rather than the time of the checkout, so python misses the change and
>>> you get all kinds of weird errors. We have taken to using lines like
>>> this for our tests:
>>>
>>>    make -j && find . -name \*.pyc -delete && bin/samba_kcc [...]
>>
>> I have done some cleanup all around and found old stuff still lingering
>> in /usr/local/samba from older installations. I am used to make upgrade
>> and test "in place" (and it is what is also documented on the wiki [1]),
>> however it does not delete old stuff like ./sbin/swat, ./bin/smbclient4,
>> etc. I cleaned up the whole thing and now I have a nice samba4.3Git tree
>> with up to date .pyc files.
>>
>> Moreover I actually also did messed up with my git tree, so I got my
>> things back together. Now I don't have spurious python error.
>>
>> So here is where I am currently:
>>
>> using old kcc :
>> * in "AD site ands services" mmc, delete inbound repls on both servers
>> * restart samba on both server (to be sure to trigger the builtin kcc)
>> * wait 20s. and both connexion are back automatically
>> * trigger samba_kcc in that situation : no change (I wasn't expecting
>> any)
>>
>> switching to new samba_kcc in smb.conf
>> * in "AD site ands services" mmc, delete inbound replications objects on
>> both servers
>> * restart samba on both server
>> * trigger samba_kcc by hand, wait a few minutes : no kcc connections
>> objects get created, per partition inbound/outbound neighbors are not
>> all there
>> however:
>> * creating by hand the kcc connexion objects in mmc get everything back
>> in order
>>
>> So I didn't manage to get it working, but at least it doesn't make funny
>> things anymore. I'll try to dig deeper tomorow.
>>
>> Cheers
>>
>> Denis
>>
>> [1] https://wiki.samba.org/index.php/Updating_Samba#Update_process
>>
>>
>>
>>>
>>>> By the way, looking a the kcc computation logic code, it is really mind
>>>> bloggling! Keep on the good job!
>>>
>>> It is fair to say that Garming and I frequently suffered from boggled
>>> minds in the course of this work.
>>>
>>> cheers,
>>> Douglas
>>>
>>
>

-- 
Denis Cardon
Tranquil IT Systems
Les Espaces Jules Verne, bâtiment A
12 avenue Jules Verne
44230 Saint Sébastien sur Loire
tel : +33 (0) 2.40.97.57.55
http://www.tranquil-it-systems.fr