ber_read_OID_String patch

Stefan (metze) Metzmacher metze at samba.org
Fri Sep 4 01:59:55 MDT 2009


Hi Kamen,
> On 9/3/2009 5:06 PM, Stefan (metze) Metzmacher wrote:
>> Kamen Mazdrashki schrieb:
>>   
>>> On Wed, Sep 2, 2009 at 11:43, Kamen Mazdrashki wrote:
>>>     
>>>> I've hit a strange problem while testing DRSUAPI interface against W2K3
>>>> server - NDR pull layer failed to decode drsuapi_DsReplicaOIDMapping
>>>> structure.

I assume you're using a W2K3 server with the schema modifications from
Exchange? We had a report of a related bug before (and haven't fixed it yet)

With plain w2k, w2k3, w2k3r2, w2k8 and w2k8r2 we don't hit any
marshaling problems. There we can correctly decode the binary oid
into strings.

The only thing we need to do is to extend our parser to cope with the
strange formated binary oids, which happen when the exchange schema
extentions are used.

And it's likely that the patch you've send already fixes this problem,
but I need the output of smbtorture with -d 101, to verify that.

>>>> After some digging, it turns out to be  ber_read_OID_String() function
>>>> fault for the failure.
>>>> And here is the strange part - it always fail to decode BER encoded OID
>>>> with __ndr_size=12 bytes. According to documentation (ref.
>>>> http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf ->
>>>> 8.19) this function works correctly. Basically every subidentifier is
>>>> encoded as a series of octets with bit 8 set to 1 and last octet with
>>>> bit 8 set to 0. So far so good. Unfortunately, W2K3 server sends BER
>>>> encoded OID where last identifier is encoded as 0x81 for example - btw,
>>>> this always happens when BER encoded value is of size 12 (as I
>>>> mentioned earlier).
>>>>
>>>>        
>>> After reading the docs some more I've come to another possible reason
>>> for the above-mentioned problem:
>>> 4. Current implementation for PrefixMap for DRSUAPI is not correct.
>>>      
>> The prefixMap attribute on the schema head object is never exposed via
>> any protocol directly (you can't get it via LDAP).
>>
>> That's why we can store it as we like.
>> And we use oid strings instead of truncated binary oids.
>>
>>    
> Actually, what I meant here was the PrefixMap composed for replica. I.e.
> drsuapi_DsReplicaOIDMapping_Ctr structure. I don't know how the
> internally used PrefixMap is being created or used. But I guess during
> replica creation internal PrefixMap is just being copied to
> drsuapi_DsReplicaOIDMapping_Ctr structure?
> 
> 
>>> I hope I am really wrong here :(. Otherwise code base to be altered
>>> and tested is a LOT.
>>> Please, take a look at
>>> http://msdn.microsoft.com/en-us/library/cc228445%28PROT.10%29.aspx
>>> As I understood this - BER encoded OIDs in PrefixMap are not
>>> complete. In order to completely decode an OID we need to add ATTRTYP
>>> value to the corresponding PrefixMapEntry.
>>>      
>> I think we should just use a special function maybe called
>> ber_read_trunated_OID_String() ) to correctly decode/encode
>> the binary oid value to a string in
>> ndr_pull_drsuapi_DsReplicaOID/ndr_push_drsuapi_DsReplicaOID.
>>
>>    
> I can't see how we could implement it that way. According to MS docs,
> partial OID are partial in their binary form.

You should not read the MS docs literally...

> Thus the algorithm is:
> 1. when creating replica prefix map entry (drsuapi_DsReplicaOIDMapping):
> 
>    - BER encode attribute OID to binary-OID
>    - based on the last string-OID subidentier value, truncate last 1 or
>    2 bytes from binary-OID
>    - NOTE: truncated 1 or 2 bytes are used to create LOWORD for
>    drsuapi_DsReplicaAttribute->attid
> 
> 2. when decoding replica received, attribute string-OID for attribute in
> drsuapi_DsReplicaAttribute structure should be decoded :
> 
>    - get LOWORD of drsuapi_DsReplicaAttribute->attid and locate
>    PrefixMap entry based on HIWORD(drsuapi_DsReplicaAttribute->attid)
>    - restore partial-binary-OID from PrefixMap entry found based on the
>    LOWORD(drsuapi_DsReplicaAttribute->attid) value
>    - BER decode binary-OID to a string-OID

I think you're mixing two things:

- the transmission of the prefix table via the
  drsuapi_DsReplicaOIDMapping_Ctr structure,
  which has nothing to do with actual ATTRTYP values.
  This is just an array of oid prefixes and there're
  integer prefix. The difference is that windows
  uses the oid prefixes in binary form and samba
  uses a string form.

- The conversation algorithm between a oid to an ATTRTYP
  and the reverse direction. I assume the confusion is based
  on the fact that windows seems to do the mapping based on
  binary oids, while we do the mapping based on string oids.

  See dsdb_map_oid2int() and dsdb_map_int2oid() for how we
  implement the same algorithms as windows without literally
  doing the same.

> The above algorithm splits encoded binary-OID in two distinct places -
> so I don't think we are able to fully decode PrefixMap during
> drsuapi_DsReplicaOIDMapping_Ctr structure decoding (unmarshaling).
> Please look at the first part of my first e-mail which I left here for
> reference - in case of Exchange installation prefix map could not be
> decoded completely.
>>> This leads to following consequences:
>>> 1. attids enum (drsuapi_DsAttributeId) are kind of obsolete - or,
>>> more properly said, those values are correct only for certain cases
>>>      
>> It's just a uint32, the reason we have some special hardcoded values in
>> our idl is that some of the prefix mappings are also hardcoded.
>> See the table in
>> http://msdn.microsoft.com/en-us/library/cc228445%28PROT.10%29.aspx
>> This makes the coding of the DsAddEntry call easier, however we could
>> also change that and always create prefix tables with the hardcoded
>> values and do dynamic mapping.
>>
>>    
> I agree - actually I admire your work on replication. You did fabulous
> work to decode the idl just using data on the wire!!
>>> 2. decoding of PrefixMap received during replication must be
>>> postponed to the point when we decode Attrubute OIDs received in replica
>>>      
>> I don't understand why this would be needed, we just need a parser for
>> the truncated binary oids.
>>
>>    
> Please, see the section for replica prefix map entry creation - I think
> we just don't have the whole information to do this.
>>> 3. PrefixMap creation is flexible and depends on the set of objects
>>> being replicated. As it depends on current Schema, we can re-create
>>> it when schema is updated.
>>>      
>> Yes, that's why we have to check that the
>> drsuapi_DsReplicaOIDMapping_Ctr matches between the source dsa and us
>> (as destination dsa) before replicating naming contexts other than the
>> schema partition. If we hit a mismatch we first need to replicate the
>> schema partition (maybe we need to directly replicate from the schema
>> master, but we have to test that).
>>
>>    
> Frankly said, I am not fully aware how replication works - in my mind,
> full replication cycle should work in order: replicate Schema, replicate
> Configuration, replicate Domain partition.

The order isn't fixed and it depends on the replication topology.

> Nevertheless, I understand we need to check if the PrefixMap in replica
> is consistent with what we have in Schema, as the schema on source-dsa
> could be changed during Schema-Configuration-Domain replication cycle.

See the dsdb_verify_oid_mappings_drsuapi() function and where it's used.

I think if the schema changed we should replace the prefix mappings we
have store. But I think we don't do that currently.

> Btw, what I am planing to do following two days is to create few
> replica-consistency tests to fully test all PrefixMap issues I had already.
> Another thing I am interested in is - does the PrefixMap in replica
> changes based on objects being replicated or it is constant no matter
> what object/attributes server send us in Replica.

As far as I know it only changes when the schema has changed and
a oid was used and the prefix of this oid was not already in the prefix
table of the schema fsmo master.

BTW: I don't understand what you mean with 'replica', it seems you mean
'replication' instead?


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20090904/8adbc357/attachment.pgp>


More information about the samba-technical mailing list