ber_read_OID_String patch

[Kamen Mazdrashki]

On 9/3/2009 5:06 PM, Stefan (metze) Metzmacher wrote:
> Kamen Mazdrashki schrieb:
>    
>> On Wed, Sep 2, 2009 at 11:43, Kamen Mazdrashki wrote:
>>      
>>> I've hit a strange problem while testing DRSUAPI interface against W2K3
>>> server - NDR pull layer failed to decode drsuapi_DsReplicaOIDMapping
>>> structure.
>>> After some digging, it turns out to be  ber_read_OID_String() function
>>> fault for the failure.
>>> And here is the strange part - it always fail to decode BER encoded OID
>>> with __ndr_size=12 bytes. According to documentation (ref.
>>> http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf ->
>>> 8.19) this function works correctly. Basically every subidentifier is
>>> encoded as a series of octets with bit 8 set to 1 and last octet with
>>> bit 8 set to 0. So far so good. Unfortunately, W2K3 server sends BER
>>> encoded OID where last identifier is encoded as 0x81 for example - btw,
>>> this always happens when BER encoded value is of size 12 (as I
>>> mentioned earlier).
>>>
>>>        
>> After reading the docs some more I've come to another possible reason for the above-mentioned problem:
>> 4. Current implementation for PrefixMap for DRSUAPI is not correct.
>>      
> The prefixMap attribute on the schema head object is never exposed via
> any protocol directly (you can't get it via LDAP).
>
> That's why we can store it as we like.
> And we use oid strings instead of truncated binary oids.
>
>    
Actually, what I meant here was the PrefixMap composed for replica. I.e. 
drsuapi_DsReplicaOIDMapping_Ctr structure. I don't know how the 
internally used PrefixMap is being created or used. But I guess during 
replica creation internal PrefixMap is just being copied to 
drsuapi_DsReplicaOIDMapping_Ctr structure?


>> I hope I am really wrong here :(. Otherwise code base to be altered and tested is a LOT.
>> Please, take a look at http://msdn.microsoft.com/en-us/library/cc228445%28PROT.10%29.aspx
>> As I understood this - BER encoded OIDs in PrefixMap are not complete. In order to completely decode an OID we need to add ATTRTYP value to the corresponding PrefixMapEntry.
>>      
> I think we should just use a special function maybe called
> ber_read_trunated_OID_String() ) to correctly decode/encode
> the binary oid value to a string in
> ndr_pull_drsuapi_DsReplicaOID/ndr_push_drsuapi_DsReplicaOID.
>
>    
I can't see how we could implement it that way. According to MS docs, 
partial OID are partial in their binary form.
Thus the algorithm is:
1. when creating replica prefix map entry (drsuapi_DsReplicaOIDMapping):

    - BER encode attribute OID to binary-OID
    - based on the last string-OID subidentier value, truncate last 1 or
    2 bytes from binary-OID
    - NOTE: truncated 1 or 2 bytes are used to create LOWORD for
    drsuapi_DsReplicaAttribute->attid

2. when decoding replica received, attribute string-OID for attribute in 
drsuapi_DsReplicaAttribute structure should be decoded :

    - get LOWORD of drsuapi_DsReplicaAttribute->attid and locate
    PrefixMap entry based on HIWORD(drsuapi_DsReplicaAttribute->attid)
    - restore partial-binary-OID from PrefixMap entry found based on the
    LOWORD(drsuapi_DsReplicaAttribute->attid) value
    - BER decode binary-OID to a string-OID

The above algorithm splits encoded binary-OID in two distinct places - 
so I don't think we are able to fully decode PrefixMap during 
drsuapi_DsReplicaOIDMapping_Ctr structure decoding (unmarshaling).
Please look at the first part of my first e-mail which I left here for 
reference - in case of Exchange installation prefix map could not be 
decoded completely.
>> This leads to following consequences:
>> 1. attids enum (drsuapi_DsAttributeId) are kind of obsolete - or, more properly said, those values are correct only for certain cases
>>      
> It's just a uint32, the reason we have some special hardcoded values in
> our idl is that some of the prefix mappings are also hardcoded.
> See the table in
> http://msdn.microsoft.com/en-us/library/cc228445%28PROT.10%29.aspx
> This makes the coding of the DsAddEntry call easier, however we could
> also change that and always create prefix tables with the hardcoded
> values and do dynamic mapping.
>
>    
I agree - actually I admire your work on replication. You did fabulous 
work to decode the idl just using data on the wire!!
>> 2. decoding of PrefixMap received during replication must be postponed to the point when we decode Attrubute OIDs received in replica
>>      
> I don't understand why this would be needed, we just need a parser for
> the truncated binary oids.
>
>    
Please, see the section for replica prefix map entry creation - I think 
we just don't have the whole information to do this.
>> 3. PrefixMap creation is flexible and depends on the set of objects being replicated. As it depends on current Schema, we can re-create it when schema is updated.
>>      
> Yes, that's why we have to check that the
> drsuapi_DsReplicaOIDMapping_Ctr matches between the source dsa and us
> (as destination dsa) before replicating naming contexts other than the
> schema partition. If we hit a mismatch we first need to replicate the
> schema partition (maybe we need to directly replicate from the schema
> master, but we have to test that).
>
>    
Frankly said, I am not fully aware how replication works - in my mind, 
full replication cycle should work in order: replicate Schema, replicate 
Configuration, replicate Domain partition.
Nevertheless, I understand we need to check if the PrefixMap in replica 
is consistent with what we have in Schema, as the schema on source-dsa 
could be changed during Schema-Configuration-Domain replication cycle.

Btw, what I am planing to do following two days is to create few 
replica-consistency tests to fully test all PrefixMap issues I had already.
Another thing I am interested in is - does the PrefixMap in replica 
changes based on objects being replicated or it is constant no matter 
what object/attributes server send us in Replica.

Kamen