[patches] make upgradeprovision use replPropertyMetaData and others cools things

[Matthieu Patou]

On 24/04/2010 16:05, Stefan (metze) Metzmacher wrote:
> Hi Matthieu,
>
>    
>> Please find attached 5 patches for provision and upgradeprovision
>>
>> 0001-s4-provision-Add-information-about-provisioned-usn-r.patch
>> 0002-s4-upgrade-provision-Refactor-code-to-do-all-the-mod.patch
>> 0003-s4-Add-functions-related-to-ldb-manipulation-when-do.patch
>> 0004-s4-upgradeprovision-Inform-about-new-dns-dynamic-upd.patch
>> 0005-s4-upgradeprovision-Use-replPropertyMetaData-for-bet.patch
>>
>>
>> The first one is to add the @PROVISION object to the base sam.ldb, this
>> object for the moment store the lastProvisionUSN attribute that indicate
>> which usn range has been touched by the provision. This information will
>> be used later on by the upgradeprovision to know which range can be
>> safely modified and which can't ...
>>
>> The second one is a refactoring of the code of upgradeprovision to allow
>> to use 1 transaction so that it's easier to track usn modification
>> during upgradeprovision and also if upgradeprovision fails it leaves the
>> database as it was before.
>>
>> The third patch is to add functions needed by upgradeprovision
>>      
> Why are the registry functions commented out?
>    
Well this is because at the beginning I had the idea to have all the 
ldb, but it turns out that path to registry ldbs is relative.
And if I open those ldbs it will create the files in the current directory.
To my mind we should have non relative path but mathias seems to think 
the opposite (and indeed at the present time having absolute path for 
registry files will break things), so the function should be more clever 
and add prefix when the path is not absolute.
For the moment I don't feel it as a huge need, but I leaved the comments 
so that it will allow me to remember to do it eventually.

Maybe the comments should be removed ?
>    
>> The forth one is to inform about the new dns changes and the fact that
>> maybe the sysadmin should do something.
>>
>> The last one is the introduction of replPropertyMetaData to drive the
>> upgrade, the idea here is to modify only the current host is the owner
>> of the attribute (originate_invocationid) and if the attribute was
>> modified during a provision or a previous upgradeprovision. This
>> information will only be available on new provision but there is a
>> script that I have (and that I can share but in the same time I didn't
>> wan't to spend ages on the cleaning) that creates an ldif file ready to
>> be integrated in the current provision to populate the range (it use an
>> educated guess to do so).
>>      
> Why do we have special cases for "member"?
>
>    
Well, for the pre-repl mode, it's because the upgrade tries to modify as 
less as possible but when upgrading different version it turns out that 
updating the "member" was a need, otherwise in the function 
handle_special_case when we have a usn it's just a left over as links 
are managed a bit before, I'll correct this dead code. Normally all the 
link attribute are handled in handle_links.
> Do we have some documentation that describes the logical steps
> do do on the upgrade? At least for me it's hard to find out how this is
> supposed to work.
>    
Well the code it self is documented but I can add some docs either in 
the program itself or in a separate file.
> I'm not sure it's correct to only "fix" local generated values.
>    
I'm not sure I understand what do you mean.
Do you mean that maybe it's a better idea to make a new provision and 
then force a replication from the old to the new provision ?
> It might be better to generate a new invocationId for the database
> (as the restore on from backup on windows does) and important the old
> stuff as replicated updates.
>    
Well a few reason not to do so :

* I suppose that the MS model is quite stable, for the moment it's still 
not the case for samba provision (although we have less changes lately)
* I suppose you can't restore a DC when there is already one running DC 
for the domain (or put it in another way: restore is a way to recreate a 
domain and if you still have a DC then it's pointless to do so as you 
just need to make a dcpromo to add a new DC), here we should be able to 
run upgrade provision with a provision with more than 1 DC (it was the 
sense of the modification that tridge suggested)
* I'm hopping to have time to focus on openldap backend soon, for them 
the replication through drs was (is?) not an option
* The goal is also to be able to upgrade from older provision where all 
this stuff is not necessarily available
* If you change the invocationID it's more difficult after to understand 
which attribute/object you can override because it has been 
created/modified by (upgrade)provision as the rule for the core update 
is allow a modification if the usn has been modified during 
provision/upgradeprovision and if the current DC is the owner of this 
modification.

Matthieu
> metze
>
>