[cifs-protocol] [REG: 117070616001855] [MS-DRSR] Calling GetNCChanges with DRS_GET_TGT flag
Edgar Olougouna
edgaro at microsoft.com
Thu Jul 13 21:46:55 UTC 2017
Tim,
Thank you for your clarification. I see your concern regarding a hanging link to an unknown object.
Please find Section references in MS-ADTS and a link to a blog post on AD internals that might help. The key point is that a linked attribute needs to reference a physical object in the database. However, attribute values may reference objects in an NC for which no NC replica is present on the local server.
When you have a valid dsname and the referenced object does not exist in the local data table, the implementation creates a valued representation of the object, commonly called phantom record, and the database identifier of that record is used in lieu of an actual object in the local database. This is an implementation-specific way of managing references to objects which the local server does not hold a replica.
After source code review, the algorithms in “[MS-DRSR] 4.1.10.6.1 ProcessGetNCChangesReply, and 4.1.10.6.14 ProcessLinkValue” appear to reflect Windows implementation.
Think of the DRS_GET_TGT in case we failed to apply the update because we had a recycled target, the client asks again for that packet to include updates to the link target.
“Reference update” is one of the background tasks described in MS-ADTS 3.1.1.6.2 (see link and related Sections). It helps update references, including when an object in the NC replica not present on the server is modified or deleted.
NOTE: I found the following blog resourceful as it describes some AD internals on tables implemented in Windows AD. Note that the blog provides implementation-specific details.
MCM: Core Active Directory Internals
https://blogs.technet.microsoft.com/askpfeplat/2012/07/22/mcm-core-active-directory-internals/
[MS-ADTS]
3.1.1.6.2 Reference Update https://msdn.microsoft.com/en-us/library/dd240022.aspx
References
* Variable: dsname<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_4d5e1f08-aa00-4dde-9411-7dd6e09ed85a>
* LDAP attributes: dNReferenceUpdate<https://msdn.microsoft.com/en-us/library/cc219845.aspx>.
* LDAP classes: infrastructureUpdate<https://msdn.microsoft.com/en-us/library/cc221881.aspx>.
* Glossary terms: dsname, Infrastructure FSMO master, NC replica<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_325d116f-cdbe-4dbd-b7e6-769ba75bf210>, tombstone<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_9d8e0963-13fa-4e19-a97f-7ce6bc90d20f>, GC.
* IDL_DRSVerifyNames method: see [MS-DRSR]<https://msdn.microsoft.com/en-us/library/cc228086.aspx> section 4.1.27<https://msdn.microsoft.com/en-us/library/dd207886.aspx>.
* Well-known Objects
In AD DS<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_2e72eeeb-aee9-4b0a-adc6-4476bacf5024>, attributes<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_108a1419-49a9-4d19-b6ca-7206aa726b3f> of attribute syntax<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_811ea26f-62cc-472e-9aca-9448831f16d8> Object (DS-DN), Object(DN-String), Object(DN-Binary), Object(Access-Point) and Object(OR-Name) can have attribute values that reference objects<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_8bb43a65-7a8c-4585-a7ed-23044772f8ca> in an NC<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_784c7cce-f782-48d8-9444-c9030ba86942> for which no NC replica is present on the server. The server does not get a replicated update<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_2a923099-db0a-4932-af28-4354601e85c4> when an object in the NC replica not present on the server is modified or deleted. In such a case, references to such objects will remain to an old dsname on the server. In order to update<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_b242435b-73cc-4c4e-95f0-b2a2ff680493> these kinds of references, a background task called reference update is run at regular intervals. By default, each reference is examined every two days.
The reference update task is not run on a Global Catalog.
If the Recycle Bin<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_54624800-58f4-45e9-90bf-c9b52dcf98f3> optional feature<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_785b66f1-22b3-450f-97aa-a24a39d04d47> is not enabled and the Infrastructure FSMO master is not a global catalog, then the reference update task is run only on the Infrastructure FSMO master.
If the Recycle Bin optional feature is enabled, every DC<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_76a05049-3531-4abd-aec8-30e19954b4bd> that is not also a global catalog runs the reference update task.
The reference update task does processing as follows:
For each object P in each NC replica on the server do the following:
* Let S be the set of all attributes of P with attribute syntax Object(DS-DN), Object(DN-String), Object(DN-Binary), Object(OR-Name) and Object(Access-Point).
* For each attribute A in set S and for each value V of A do the following:
* If there exists an object with dsname V in any NC replica on this DC, then skip this value V.
* If attribute syntax of A is Object(DS-DN) then let G be P.A.V.guid_value. Let D be P.A.V.dn.
* Otherwise, let G be P.A.V.object_DN.guid_value. Let D be P.A.object_DN.dn.
* If the Recycle Bin optional feature is not enabled:
* Retrieve the dsname N of object with objectGUID<https://msdn.microsoft.com/en-us/library/cc221017.aspx> G from a GC by calling method IDL_DRSVerifyNames. IDL_DRSVerifyNames is explained in [MS-DRSR] section 4.1.27.
* If N!name<https://msdn.microsoft.com/en-us/library/cc220701.aspx> ≠ D then create an infrastructureUpdate object I in the well-known infrastructure update container<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_c3143e71-2ada-417e-83f4-3ef10eff2c56> (see section 6.1.1.4<https://msdn.microsoft.com/en-us/library/cc223663.aspx>). Set I!dNReferenceUpdate to N. Delete I immediately to turn it to a tombstone.
Creation of an infrastructureUpdate object K with attribute dNReferenceUpdate will trigger an update of all references to dsnames corresponding to K!dNReferenceUpdate, as explained in section 3.1.1.5.2.4<https://msdn.microsoft.com/en-us/library/cc223446.aspx>.
* If the Recycle Bin optional feature is enabled:
* Retrieve the dsname N and the value Vgc of the isRecycled<https://msdn.microsoft.com/en-us/library/dd357315.aspx> attribute of object with objectGUID G from a GC by calling method IDL_DRSVerifyNames. IDL_DRSVerifyNames is explained in [MS-DRSR] section 4.1.27.
* If Vgc is true and attribute A is a linked attribute, remove value V from attribute A. This removal is not replicated to any other DCs.
* If N!name ≠ D then replace value V of attribute A with N!name. This replacement is not replicated to any other DCs.
* If attribute A is a link value<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_659e8352-a6db-4752-8c05-4b21c602f238> and the RDN<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_22198321-b40b-4c24-b8a2-29e44d9d92b9> of N!name is a delete-mangled RDN (see section 3.1.1.5.5<https://msdn.microsoft.com/en-us/library/cc223480.aspx>), the value V is to be treated as a linked value to or from a deleted-object<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_d9c9e99f-74f1-483e-bcb1-310e75ff1344>. That is, the value is not generally visible to LDAP<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_45643bfb-b4c4-432c-a10f-b98790063f8d> clients unless the LDAP_SHOW_DEACTIVATED_LINK_OID control is used.
* If attribute A is a link value and the RDN of N!name is not a delete-mangled RDN (see section 3.1.1.5.5), the value V is to be treated as a normal linked value. That is, the value is generally visible to LDAP clients.
3.1.1.3.3.7 checkPhantoms https://msdn.microsoft.com/en-us/library/cc223316.aspx
This operation requests that the reference update<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_b242435b-73cc-4c4e-95f0-b2a2ff680493> task (see section 3.1.1.6.2<https://msdn.microsoft.com/en-us/library/dd240022.aspx>) be immediately performed on the DC<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_76a05049-3531-4abd-aec8-30e19954b4bd>. During the operation, if the referential integrity on any of the objects<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_8bb43a65-7a8c-4585-a7ed-23044772f8ca> is found to be incorrect and it cannot be corrected, then the operation returns an error and does not process any of the remaining objects. This task runs periodically; on a correctly functioning DC, there is no need to run it explicitly. The requester must have the "DS-Check-Stale-Phantoms" control access right<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_42f6c9e0-a2b3-4bc3-9b87-fdb902e5505e> on the nTDSDSA<https://msdn.microsoft.com/en-us/library/cc221718.aspx> object for the DC.
No action is taken if the Recycle Bin<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_54624800-58f4-45e9-90bf-c9b52dcf98f3> optional feature<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_785b66f1-22b3-450f-97aa-a24a39d04d47> is not enabled and the operation is performed against a DC that does not own the Infrastructure Master FSMO.
No action is taken if the operation is performed against a DC that is a global catalog.
The type of modification can be add or replace, and the values specified in the LDAP<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_45643bfb-b4c4-432c-a10f-b98790063f8d> modify operation do not matter.
The following shows an LDIF sample that performs this operation.
dn:
changetype: modify
add: checkPhantoms
checkPhantoms: 1
-
6.1.5.5 Infrastructure FSMO Role https://msdn.microsoft.com/en-us/library/cc223753.aspx
When an object<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_8bb43a65-7a8c-4585-a7ed-23044772f8ca> in one domain<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_b0276eb2-4e65-4cf1-a718-e0920a614aca> is referenced by another object in another domain, it represents the reference as a dsname<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_4d5e1f08-aa00-4dde-9411-7dd6e09ed85a>. There is one Infrastructure FSMO role per domain and application NC<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_53fee475-b07f-45e1-b4b7-c7ac0c1e7f6a> in a directory<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_49ce3946-04d2-4cc9-9350-ebcd952b9ab9>.
If all the domain controllers<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_76a05049-3531-4abd-aec8-30e19954b4bd> in a domain also host the GC, then all the domain controllers have the current data, and it is not important which domain controller owns the Infrastructure Master (IM) role. See section 3.1.1.5<https://msdn.microsoft.com/en-us/library/cc223433.aspx> for more information about the Infrastructure Master.
When the Recycle Bin<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_54624800-58f4-45e9-90bf-c9b52dcf98f3> optional feature<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_785b66f1-22b3-450f-97aa-a24a39d04d47> is not enabled, the Infrastructure FSMO role owner is the DC responsible for updating a cross-domain object reference<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_3ca938ae-c14f-4f59-8a7d-daca9f76db4e> in the event that the referenced object is moved, renamed, or deleted. In this case, the Infrastructure Master role must be held by a domain controller that is not a GC server<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_a5a99ce4-e206-42dc-8874-e103934c5b0d>. If the Infrastructure Master runs on a GC server, it will not update<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_b242435b-73cc-4c4e-95f0-b2a2ff680493> object information, because it does not contain any references to objects that it does not hold. This is because a GC server holds a partial replica<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_ea02e669-2dda-460c-9992-b12a23caeeac> of every object in the forest<https://msdn.microsoft.com/en-us/library/cc223126.aspx#gt_fd104241-4fb3-457c-b2c4-e0c18bb20b62>.
When the Recycle Bin optional feature is enabled, every DC is responsible for updating its cross-domain object references in the event that the referenced object is moved, renamed, or deleted. In this case, there are no tasks associated with the Infrastructure FSMO role, and it is not important which domain controller owns the Infrastructure Master role.
[MS-DRSR]
4.1.10.6.1 ProcessGetNCChangesReply
/* Process link value updates. */
linkValueCount := 0
while (ulResult = 0) and (linkValueCount < msgReplyNative.cNumValues)
ulResult := ProcessLinkValue(
msgReplyNative.rgValues[linkValueCount],
msgReplyNative.pNC^,
prefixTable,
msgIn.ulFlags,
msgIn.ulMoreFlags)
linkValueCount := linkValueCount + 1
endwhile
if (ulResult = ERROR_DS_DRA_MISSING_PARENT) then
Send IDL_DRSGetNCChanges message again with the same input
parameters specified in msgIn but this time with msgIn.ulFlags
containing DRS_GET_ANC field set. It is an error for this
condition to occur if (DRS_GET_ANC in msgIn.ulFlags) is true
else if (ulResult = ERROR_DS_DRA_RECYCLED_TARGET) then
Send IDL_DRSGetNCChanges message again with the same input
parameters specified in the msgIn but this time with msgIn.ulMoreFlags
containing DRS_GET_TGT field set.
else if (msgIn.ulExtendedOp = 0) then
/* Not an extended operation. Update "watermark" information. */
UpdateRepsFrom(
rf,
msgReplyNative,
dsaServer,
ulResult)
if (ulResult = 0) and (msgReplyNative.fMoreData = false) then
UpdateUTDandPAS(
msgReplyNative,
msgIn.partialAttrSetEx^)
endif
endif
4.1.10.6.14 ProcessLinkValue
. . .
newAttributeValue = ValueFromATTRVAL(
sourcePrefixTable, Syntax(attribute), replValInf.pAval)
targetObject := GetDSNameFromAttrVal( replValinf.attrTyp, replValInf.pAval)
if (targetObject = null)
return ERROR_DS_INVALID_ATTRIBUTE_SYNTAX
if ((IsRecycleBinEnabled() and targetObject!isRecycled) or
(not IsRecycleBinEnabled() and targetObject!isDeleted)) then
if (DRS_GET_TGT in ulMoreFlags) then
/* nothing to do */
return 0
else
return ERROR_DS_DRA_RECYCLED_TARGET
endif
endif
if (not attributeValue = null) then
/* Remove the old attribute value. */
RemoveAttrVal(updateObject, attribute, attributeValue)
endif
SetAttrVal(updateObject, attribute, newAttributeValue)
. . .
Thanks,
Edgar
From: Tim Beale [mailto:timbeale at catalyst.net.nz]
Sent: Sunday, July 9, 2017 4:44 PM
To: Edgar Olougouna <edgaro at microsoft.com>; 'abartlet at samba.org' <abartlet at samba.org>
Cc: cifs-protocol at lists.samba.org; MSSolve Case Email <casemail at microsoft.com>
Subject: Re: [cifs-protocol] [REG: 117070616001855] [MS-DRSR] Calling GetNCChanges with DRS_GET_TGT flag
Hi Edgar,
>> Request more detail in case of cross-partition target
We understand that it requires implementation-specific behaviour to resolve an unknown target object when the linked attribute spans partitions. This was answered for us in 117060115827644.
>> Target object is only recycled, as opposed to Target-object is unknown
>> (linked-attribute and you don't know the target, you send DRS_GET_TGT)
This is what I'm confused about. I inferred from the MS_DRSR Docs that the client uses DRS_GET_TGT if it receives a linked attribute with an unknown target object. However, the answers we received for 117060815865307 seemed to imply that DRS_GET_TGT is only used if the target object is deleted/recycled (as opposed to simply being a new/unknown object). Could you please clarify this - should the client use it in either case, or is it only the deleted/recycled case where the flag is actually required?
>> Implementation-specific statement regarding client using DRS_GET_TGT
Basically it's the following block of code in 4.1.10.6.14 ProcessLinkValue() that I find confusing/misleading.
targetObject = GetDSNameFromAttrVal( replValinf.attrTyp, replValInf.pAval)
if ((IsRecycleBinEnabled() and targetObject!isRecycled) or
(not IsRecycleBinEnabled() and targetObject!isDeleted)) then
if (DRS_GET_TGT in ulMoreFlags) then
/* nothing to do */
return 0
else
return ERROR_DS_DRA_RECYCLED_TARGET
endif
I don't think the proposed Docs change for 117060815865307 makes this any easier to understand. Documentation-wise, I think there's room for the following minor improvements:
1. Section 3.4.5 describes the ! notation. However, I couldn't see anything that describes what happens if the object specified by the DN doesn't exist. I would assume the attribute returned would be null.
2. If the target object doesn't exist, the current code looks like the 4.1.10.6.14 block will fall through and add a hanging link to an unknown object. Is that the intended behaviour? A comment stating that would be nice.
3. If there is other implementation-specific behaviour in the 4.1.10.6.14 block, then a comment stating "The actual behaviour here may be implementation-specific" would be nice.
4. I also just noticed the above block is missing an endif.
Thanks for your help,
Tim
On 07/07/17 04:51, Edgar Olougouna via cifs-protocol wrote:
Please provide any additional write-up as necessary. We will review this and follow-up.
Implementation-specific statement regarding client using DRS_GET_TGT
Request more detail in case of cross-partition target
Target object is only recycled, as opposed to Target-object is unknown
(linked-attribute and you don't know the target, you send DRS_GET_TGT)
Thanks,
Edgar
_______________________________________________
cifs-protocol mailing list
cifs-protocol at lists.samba.org<mailto:cifs-protocol at lists.samba.org>
https://lists.samba.org/mailman/listinfo/cifs-protocol<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.samba.org%2Fmailman%2Flistinfo%2Fcifs-protocol&data=02%7C01%7Cedgaro%40microsoft.com%7C5f4eebde07da4724ff5a08d4c7139b92%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636352334412827688&sdata=Ef66txw485K2nCi3tm%2FF60ZWF4Wyds3h0qCmyAOEPJc%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/cifs-protocol/attachments/20170713/3ad7672f/attachment-0001.html>
More information about the cifs-protocol
mailing list