[PATCH] GUID index for LDB

[Stefan Metzmacher]

Am 08.09.2017 um 05:56 schrieb Andrew Bartlett:
> On Thu, 2017-09-07 at 12:03 +1200, Andrew Bartlett via samba-technical
> wrote:
>>
>> I'll put that in the release commit message and in the top of the
>> ldb_index.c file.
> 
> The attached, updated patch set includes this slab (see the patch for
> the full text):
> 
> The new 'GUID index' format is:
> -------------------------------
> 
> dn: @INDEX:NAME:DNSUPDATEPROXY
> @IDXVERSION: 3
> @IDX: <binary GUID>[<binary GUID>[...]]
> 
> The binary guid is 16 bytes, as bytes and not expanded as hexidecimal
> or pretty-printed.  The GUID is chosen from the message to be stored
> by the @IDXGUID attribute on @INDEXLIST.
> 
> If there are multiple values the @IDX value simply becomes longer,
> in multiples of 16.
> 
> The corrosponding entry is stored in a TDB record with key:
> 
> GUID=<binary GUID>
> 
> This allows a very quick translation between the fixed-length index 
> values and the TDB key, while seperating entries from other data
> in the TDB, should they be unlucky enough to start with the bytes of
> the 'DN=' prefix.  
> 
> Additionally, this allows a scope BASE search to directly find the
> record via a simple match on a GUID= extended DN, controlled via
> @IDX_DN_GUID on @INDEXLIST
> 
> Exception for special @ DNs:
> 
> @BASEINFO, @INDEXLIST and all other special DNs are stored as per the
> original format, as they are never referenced in an index and are used
> to bootstrap the database.
> 
> 
> Control points for choice of index mode
> ---------------------------------------
> 
> The choice of index and TDB key mode is made based (for example, from
> Samba) on entries in the @INDEXLIST DN:
> 
> dn: @INDEXLIST
> @IDXGUID: objectGUID
> @IDX_DN_GUID: GUID
> 
> By default, the original DN format is used.

So we're upgrading the database on first use with the new code?

My fear with this is that a simple package upgrade will make
a dc with a large database unusable for quite some time.

Can you please check the cost of an upgrade for databases with
1.) 5000 users, 5000 computers and 5000 groups
2.) 20000 users, 20000 computers and 20000 groups
3.) with the numbers of the largest known customer size

I guess rewriting the whole database consumes quite some cpu
and also memory. A server may run out of memory while doing this
as we need more than twice the size of all sam.ldb* databases together.

I think I'd prefer making the switch for existing databases an
explicit task for the admin.

Thanks!
metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170908/5ca51fb2/signature.sig>