[PATCH] do a partial replication with drs replicate --local

[Stefan Metzmacher]

Hi Bob,

> Attached are patches after my team's had a look at them; with some minor
> changes.

Thanks! Please change some minor things in py_dsdb_load_udv_v2():
- Please initialize all pointer variables with NULL
- cursor_ctr is unused
- if (!py_cursor) should be if (py_cursor == NULL)

Then it's Reviewed-by: me

Thanks!
metze

> On 09/02/17 15:10, Bob Campbell wrote:
>> Hi Metze -
>>
>> I've changed the attached patches by adding python bindings for
>> dsdb_load_udv_v2, and using that in replicate --local.
>>
>> Thanks,
>> Bob
>>
>> On 08/02/17 23:41, Stefan Metzmacher wrote:
>>> Hi Bob,
>>>
>>> no the logic is not ok yet.
>>>
>>> Am 08.02.2017 um 02:01 schrieb Bob Campbell:
>>>>  
>>>> -
>>>>      source_dsa_invocation_id = misc.GUID(self.samdb.get_invocation_id())
>>>>      dest_dsa_invocation_id = misc.GUID(self.local_samdb.get_invocation_id())
>>>>      destination_dsa_guid = self.ntds_guid
>>>>  
>>>> +    # If we can't find an upToDateVector, replicate fully
>>>> +    hwm = drsuapi.DsReplicaHighWaterMark()
>>>> +    hwm.tmp_highest_usn = 0
>>>> +    hwm.reserved_usn = 0
>>>> +    hwm.highest_usn = 0
>>>> +
>>>> +    udv = None
>>>> +    if not full_sync:
>>>> +        res = self.local_samdb.search(base=NC, scope=ldb.SCOPE_BASE,
>>>> +                                      attrs=["replUpToDateVector", "repsFrom"])
>>>> +        if "repsFrom" in res[0]:
>>>> +            for reps_from_packed in res[0]["repsFrom"]:
>>>> +                reps_from_obj = ndr_unpack(drsblobs.repsFromToBlob, reps_from_packed)
>>>> +                if reps_from_obj.ctr.source_dsa_invocation_id == source_dsa_invocation_id:
>>>> +                    hwm = reps_from_obj.ctr.highwatermark
>>>> +
>>>> +        if "replUpToDateVector" in res[0]:
>>>> +            udv_packed = res[0]["replUpToDateVector"][0]
>>>> +            udv_obj = ndr_unpack(drsblobs.replUpToDateVectorBlob, udv_packed)
>>>> +            udv_ctr = udv_obj.ctr
>>>> +
>>>> +            udv = drsuapi.DsReplicaCursorCtrEx()
>>>> +            udv.version = 2
>>>> +            udv.reserved1 = 0
>>>> +            udv.reserved2 = 0
>>>> +            cursors = []
>>>> +
>>>> +            highest_usn = self.local_samdb.sequence_number(ldb.SEQ_NEXT)
>>>> +            found = False
>>>> +            for cursor in udv_ctr.cursors:
>>>> +                udv_ex_cursor = drsuapi.DsReplicaCursor()
>>>> +                udv_ex_cursor.source_dsa_invocation_id = cursor.source_dsa_invocation_id
>>>> +                udv_ex_cursor.highest_usn = cursor.highest_usn
>>>> +                # Update the cursor if it's ours
>>>> +                if cursor.source_dsa_invocation_id == source_dsa_invocation_id:
>>>> +                    udv_ex_cursor.highest_usn = highest_usn
>>>> +                    found = True
>>>> +                cursors.append(udv_ex_cursor)
>>>> +
>>>> +            # If we didn't find our cursor, add a new one for us
>>>> +            if not found:
>>>> +                our_udv_cursor = drsuapi.DsReplicaCursor()
>>>> +                our_udv_cursor.source_dsa_invocation_id = source_dsa_invocation_id
>>>> +                our_udv_cursor.highest_usn = highest_usn
>>>> +                cursors.append(our_udv_cursor)
>>>> +
>>>> +            udv.cursors = cursors
>>>> +            udv.count = len(cursors)
>>>> +
>>> It needs to be something like this:
>>>
>>>
>>>     highest_usn = self.local_samdb.sequence_number(ldb.SEQ_NEXT)
>>>
>>>     # If we can't find an upToDateVector, replicate fully
>>>     hwm = drsuapi.DsReplicaHighWaterMark()
>>>     hwm.tmp_highest_usn = 0
>>>     hwm.reserved_usn = 0
>>>     hwm.highest_usn = 0
>>>
>>>     udv = None
>>>     if not full_sync:
>>>         res = self.local_samdb.search(base=NC, scope=ldb.SCOPE_BASE,
>>>                                       attrs=["replUpToDateVector",
>>> "repsFrom"])
>>>         if "repsFrom" in res[0]:
>>>             for reps_from_packed in res[0]["repsFrom"]:
>>>                 reps_from_obj = ndr_unpack(drsblobs.repsFromToBlob,
>>> reps_from_packed)
>>>                 if reps_from_obj.ctr.source_dsa_invocation_id ==
>>> source_dsa_invocation_id:
>>>                     hwm = reps_from_obj.ctr.highwatermark
>>>
>>>         # The following logic should match dsdb_load_udv_v2()!
>>>         cursors = []
>>>         found = False
>>>
>>>         if "replUpToDateVector" in res[0]:
>>>             udv_packed = res[0]["replUpToDateVector"][0]
>>>             udv_obj = ndr_unpack(drsblobs.replUpToDateVectorBlob,
>>> udv_packed)
>>>             udv_ctr = udv_obj.ctr
>>>
>>>             for cursor in udv_ctr.cursors:
>>>                 udv_ex_cursor = drsuapi.DsReplicaCursor()
>>>                 udv_ex_cursor.source_dsa_invocation_id =
>>> cursor.source_dsa_invocation_id
>>>                 # Update the cursor if it's ours
>>>                 if cursor.source_dsa_invocation_id ==
>>> destinatoin_dsa_invocation_id:
>>>                     udv_ex_cursor.highest_usn = highest_usn
>>>                     found = True
>>>                 else:
>>>                     udv_ex_cursor.highest_usn = cursor.highest_usn
>>>                 cursors.append(udv_ex_cursor)
>>>
>>>         # If we didn't find our cursor, add a new one for us
>>>         if not found:
>>>             our_udv_cursor = drsuapi.DsReplicaCursor()
>>>             our_udv_cursor.source_dsa_invocation_id =
>>> destination_dsa_invocation_id
>>>             our_udv_cursor.highest_usn = highest_usn
>>>             cursors.append(our_udv_cursor)
>>>
>>>         todo_sort_by_guid(cursors)
>>>
>>>         udv = drsuapi.DsReplicaCursorCtrEx()
>>>         udv.version = 2
>>>         udv.reserved1 = 0
>>>         udv.reserved2 = 0
>>>         udv.cursors = cursors
>>>         udv.count = len(cursors)
>>>
>>> The main points are that our highest_usn belongs to our
>>> destination_dsa_invocation_id not to the source_dsa_invocation_id variable.
>>> and we need to create the udv even without "replUpToDateVector" in that
>>> case we just send our own cursor.
>>>
>>> And you should to sort the cursors at the end see:
>>> TYPESAFE_QSORT(*cursors, *count, drsuapi_DsReplicaCursor2_compare);
>>>
>>> Another approach would be to add python bindings for dsdb_load_udv_v2().
>>>
>>> metze
>>>
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170209/08abf48b/signature.sig>