Trying to understand upgradeprovision

[Matthieu Patou]

On 02/14/2013 11:49 PM, Andrew Bartlett wrote:
> On Thu, 2013-02-14 at 23:27 -0800, Matthieu Patou wrote:
>> On 02/13/2013 09:31 PM, Andrew Bartlett wrote:
>>> Matthieu,
>>>
>>> I've been adding tests to help me understand upgradeprovision, what it
>>> can do and what it can't do.
>>>
>>> The branch this mail is based on is here:
>>> https://git.samba.org/abartlet/samba.git/?p=abartlet/samba.git/.git;a=shortlog;h=refs/heads/samba-tool-and-acls
>>>
>>> The issue I have is that I can't find any clear description of what
>>> --full does, and how we can safely use upgradeprovision to correct the
>>> ACLs that were incorrect in our 4.0.0 release.
>> Full will not only correct file location it will also generate a
>> reference provision and add all the objects that are in the reference
>> provision but not in the live one, it will also look at missing and
>> changed attributes and add them (well not all because some shouldn't be
>> copied but you get the idea).
>>> Is it the case that without --full, we only do things so clear that we
>>> should probably do them in dbcheck instead (and have upgradeprovision
>>> run dbcheck at the end?)
>> Well dbcheck should check the ... not a kitchen-sink
> Can you clarify for me *exactly* what upgradeprovision does in each
> mode?  That is, can you expand on the above to spell out what a 'not
> full' mode does, in more detail than 'correct file locations'?
The guy who asked me about the non full mode is you so you should know 
as good as me ...

There is also a whole documentation paragraph before the main function, 
did you had a look at it ?
Looking at the code the main difference between full and the rest is 
that the full will do the update of the sam.ldb the other mode won't but 
in both case we look for the SDs that needs to be updated (those which 
are different from the ref database) and we try to update it.

>
> In terms of things I of what I would like to see dbcheck handle (and
> upgradprovison call into dbcheck for), it is things where a specific
> rule can be applied, such as "there must be a deleted objects
> container", "links must have a backlink" etc.  For these, we don't need
> the reference provision nor the provision USN range, as these rules
> always apply.
True, you still have to remember the history of why some stuff are 
present here as we had a bumpy road toward a AD like provision.

>
>>> I had been under the impression that the SD recalculation was done
>>> regardless, but when I run these commands:
>>>
>>> bin/samba-tool ldapcmp st/provision/alpha13_upgrade/private/sam.ldb
>>> st/provision/alpha13_upgrade_reference/private/sam.ldb --two domain
>>> schema --sd > /tmp/ldapcmp-upgrade_sd.txt
>>>
>>> bin/samba-tool ldapcmp st/provision/alpha13_upgrade_full/private/sam.ldb
>>> st/provision/alpha13_upgrade_reference/private/sam.ldb --two domain
>>> schema --sd > /tmp/ldapcmp-upgrade_full_sd.txt
>>>
>>> I see that the SDs are only fixed if we run with --full.  My concern is
>>> that this mode changes many, many other things about the database, and I
>>> would prefer to find a 'lighter touch' way to handle this.
>> Well the lighter way might be --fixdsacl
> I don't see a --fixdsacl at the moment, so I presume you mean you would
> patch up a new main mode, so we would have: the default mode, --fixdsacl
> and --full?
Why not, but looking more closely, it seems that for alpha13 at least 
--fixdsacl won't be sufficient as without full with don't recalculate 
them very correctly I don't know why but there must be something that 
influence greatly how we do the calculation.

The initial idea is that --full does *everything* that is needed, no 
options was more lightweight, you add also a bunch of options for the 
ntacls that maybe should be removed as they seems not very clear and we 
have a script to do it now separately, so adding --fixdsacl whould be 
for: do nothing but fixing the acls in the DS part.
>
>>> Anyway, I include the output from those commands, in the hope that you
>>> can shed some light.  To reproduce, on my branch first run 'make test
>>> TESTS=alpha13' and the directories will be produced.
>> Well the results for the --full looks pretty good, just one object is
>> not ok (maybe a bug ?)
>>
>> Comparing:
>> 'CN=ARES,OU=Domain Controllers,DC=alpha13,DC=samba,DC=corp'
>> [st/provision/alpha13_upgrade_full/private/sam.ldb]
>> 'CN=ARES,OU=Domain Controllers,DC=alpha13,DC=samba,DC=corp'
>> [st/provision/alpha13_upgrade_reference/private/sam.ldb]
>>       ACEs found only in st/provision/alpha13_upgrade_full/private/sam.ldb:
>>           (OA;;SW;DNS-Host-Name-Attributes;;DA)
>>           (OA;;SW;DNS-Host-Name-Attributes;;PS)
>>       ACEs found only in
>> st/provision/alpha13_upgrade_reference/private/sam.ldb:
>>           (OA;;SW;Validated-DNS-Host-Name;;DA)
>>           (OA;;SW;Validated-DNS-Host-Name;;PS)
>>       FAILED
> Yes, the --full mode does very well, and if you re-run the tests without
> the --sd argument, you will see that there are only a few objects that
> are incorrectly upgraded from alpha13.  (in the test, we filter out the
> attributes so I could have a test that passed for all other objects).
Well It means ihmo that we have a bug in the recalculation function as 
if we ask the function to recalculate the SD it should do it.
>
> I am looking for something that is a lighter touch however - not
> potentially rewriting any attribute and any object, when we only changed
> around 10 items in the provision template.  Do you think you could add
> and test such a mode?
I'm sorry I don't understand this phrase.

Matthieu.

-- 
Matthieu Patou
Samba Team
http://samba.org