[PATCH] Changes to indexing to handle maximum key lengths.

[Andrew Bartlett]

On Tue, 2018-02-27 at 11:44 +1300, Gary Lockyer via samba-technical
wrote:
> Attached is the completed patch set ready for review and possibly push.
> 
> As you've seen from Gamings post we've been working on adding an lmdb
> back end to ldb.  Lmdb enforces a maximum key length, currently 511
> bytes, these patches allow the current Samba indexing scheme to be used
> with an lmdb back end.
> 
> 
> Review and or comments appreciated.

G'Day Gary,

Looking over the tests, one thing that comes to mind is that there is
no way you currently prove that the key truncation is actually
happening.  I'm concerned that instead of breaking the index layer,
what could be broken is the option parsing in LDB (as this is the only
user) and these tests could be rendered ineffective.

Could you please read the actual index values and check they are set as
expected?

This will also nicely test the layout of the GUID index mode, which
would be great as well.

You are also missing a h here (ldb_tdb.h):
+        * than this lengt will be rejected.

Also, rather than use a bool here:

 static int ltdb_index_filter(struct ltdb_private *ltdb,
                             const struct dn_list *dn_list,
                             struct ltdb_context *ac,
-                            uint32_t *match_count)
+                            uint32_t *match_count,
+                            bool trustworthy_base)

I think it would be better to pass it in as 'enum key_truncation
*scope_index_truncation' from ltdb_search_indexed() to
ltdb_index_filter.

Finally, in ltdb_index_add1()
+               /*
+                * More than one entry in the DN->GUID index, which
+                * arises when the DN indexes have been truncated
+                *
+                * So need to pull the DN's to check if it's really a duplicate
+                */

That comment needs to say 'At least one existing entry...', not 'more
than one' as we are trying to add the second entry here and it may or
may not be a duplicate. 

This is a really important step towards LMDB support, and I really appreciate it.

Thanks!

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba