big pb with reindexing and ldb_dn_casefold_internal

[Matthieu Patou]

  Hello andrew and andrew !

It seems that I found an interesting pb.

While testing upgradeprovision on a copy of my production I was unable 
to reopen the database as

ltdb: tdb(private/sam.ldb.d/DC=...,DC=.....ldb): tdb_transaction_cancel: 
no transaction

schema_load_init: dsdb_set_schema() failed: 32:No such object: del_trans 
error on DC=...,DC=....: prepare_commit error on DC=....,DC=...: Failed 
to store index records in transaction commit: Other
module schema_load initialization failed
module operational initialization failed
module acl initialization failed
module descriptor initialization failed
module objectclass initialization failed
module asq initialization failed
module server_sort initializat    ion failed
module paged_results initialization failed
module lazy_commit initialization failed
module rootdse initialization failed
module samba_dsdb initialization failed

After some investigation I found that the problem is that the tdb module 
is failling in "prepare_transaction".
And after more investigation I think I found the root cause, in our 
provision we have DisplayName with ","
ie:

displayName: Patou, Matthieu

When reindexing the ltdb_delete_noindex called by 
ltdb_index_traverse_store is trying to casefold the dn, here is an 
extract of the stacktrace:
#0  ldb_dn_explode (dn=0x94bcd10) at ../lib/ldb/common/ldb_dn.c:593
#1  0xb7857773 in ldb_dn_casefold_internal (dn=0x94bcd10) at 
../lib/ldb/common/ldb_dn.c:922
#2  0xb78579da in ldb_dn_get_casefold (dn=0x94bcd10) at 
../lib/ldb/common/ldb_dn.c:974
#3  0xb782bdc0 in ltdb_key (module=0x8526ec0, dn=0x94bcd10) at 
../lib/ldb/ldb_tdb/ldb_tdb.c:146
#4  0xb782c408 in ltdb_delete_noindex (module=0x8526ec0, dn=0x94bcd10) 
at ../lib/ldb/ldb_tdb/ldb_tdb.c:356
#5  0xb7830332 in ltdb_dn_list_store_full (module=0x8526ec0, 
dn=0x94bcd10, list=0x9466ee8)
     at ../lib/ldb/ldb_tdb/ldb_index.c:203
#6  0xb783089b in ltdb_index_traverse_store (tdb=0x8527878, key=..., 
data=..., state=0x8526ec0)
     at ../lib/ldb/ldb_tdb/ldb_index.c:325
#7  0xb7b92a2d in tdb_traverse_internal (tdb=0x8527878, fn=0xb7830775 
<ltdb_index_traverse_store>, private_data=0x8526ec0,
     tl=0xbfffd75c) at ../../lib/tdb/common/traverse.c:190
#8  0xb7b92c61 in tdb_traverse (tdb=0x8527878, fn=0xb7830775 
<ltdb_index_traverse_store>, private_data=0x8526ec0)
     at ../../lib/tdb/common/traverse.c:260
#9  0xb783094b in ltdb_index_transaction_commit (module=0x8526ec0) at 
../lib/ldb/ldb_tdb/ldb_index.c:343
#10 0xb782d670 in ltdb_prepare_commit (module=0x8526ec0) at 
../lib/ldb/ldb_tdb/ldb_tdb.c:949
#11 0xb785ccfb in ldb_next_prepare_commit (module=0x8526ec0) at 
../lib/ldb/common/ldb_modules.c:666
#12 0xb76f40dc in partition_prepare_commit (module=0x8520e68) at 
../dsdb/samdb/ldb_modules/partition.c:761
#13 0xb785ccfb in ldb_next_prepare_commit (module=0x8520e68) at 
../lib/ldb/common/ldb_modules.c:666



The dn in Frame 1 is (gdb) print *dn
$125 = {ldb = 0x84e0888, special = false, invalid = false, valid_case = 
false,
   linearized = 0x94cea18 "DN=@INDEX:DISPLAYNAME:PATOU, MATTHIEU", 
ext_linearized = 0x0, casefold = 0x0, comp_num = 0,
   components = 0x853f178, ext_comp_num = 0, ext_components = 0x0}

And it's reaching  this part of the code:
                         case ',':
                                 if (escape) {
                                         *d++ = *p++;
                                         l++;
                                         escape = false;
                                         continue;
                                 }
                                 /* ok found value terminator */

                                 if ( t ) {
                                         /* trim back */
                                         d -= (p - t);
                                         l -= (p - t);
                                 }

                                 in_attr = true;
                                 in_value = false;
                                 trim = true;

which later will trigger this:

         if (in_attr || in_quote) {
                 /* invalid dn */
                 ldb_dn_mark_invalid(dn);
                 goto failed;
         }

And then will return with a failure code.


-- 
Matthieu Patou
Samba Team        http://samba.org