[PATCH] GUID index for LDB

[Andrew Bartlett]

On Thu, 2017-09-07 at 01:25 +0200, Stefan Metzmacher wrote:
> Hi Andrew,
> 
> > > So there's no performance loss of 5% for searches?
> > 
> > There might be, but I don't think so.  Our experience is that even
> > after several runs the numbers below 5% are not statistically
> > significant, as the absolute values have too much noise.  Watch how
> > the
> > 'do nothing' line moves around for an idea.  I'll mail you the full
> > results file and the absolute values graph tomorrow. 
> > 
> > > As we're now doing one more hop from the index (now via the
> > > objectGUID)
> > > to the dn.
> > 
> > We only change the cost for a base DN search really, plus the cost
> > of
> > checking the base for a subtree search.  
> > 
> > Anything in an index avoids going from a DN -> key with a casefold,
> > as
> > both the contents of the index and the key in the DB now match
> > exactly
> > (plus a prefix). 
> > 
> > So an indexed search that was:
> >  - index -> casefold -> key
> > is:
> >  - index -> key
> 
> Isn't it index -> GUID-key -> DN->key ?

No (unless you count the memcpy() for the GUID=prefix as the GUID-key
step).

> > and a base search that was:
> >  - DN -> casefold -> key
> > is:
> >  - DN -> index -> key
> 
> I don't understand that case...
> 
> Do you actually change the primary key for the records to be GUID
> based?
> I thought you would only change the index and keep the real objects
> as they're now...

Correct, I move all the objects to allow (as should have been
originally designed) the values in the index to be trivially converted
into TDB keys. 

> > We use a number of tricks to ensure we don't waste the expense of
> > the
> > casefold. 
> > 
> > > Is it expected that only some workloads are faster?
> > 
> > Yes.  It is delete from an index that hurts the most in the current
> > code (linear scan), the rest of the benefit comes from having a
> > smaller
> > index overall, reducing the memcpy time in the read and transaction
> > commit. 
> > 
> > > Do these numbers already include the BINARY_SEARCH patches?
> > 
> > Yes.
> > 
> > > > This series also passes a full make test.  It also showed some
> > > > flapping
> > > > tests, so I plan to chase those down and I look forward to a
> > > > positive
> > > > review!
> > > 
> > > I guess you'll resort the commits so that the version bump
> > > happens
> > > at the end just before the final patch that activates it in
> > > Samba?
> > 
> > Yes, that is the plan.
> > 
> > > Can you also run an autobuild without the activation in Samba,
> > > so that we're sure we don't insert regressions for possible
> > > backports?
> > 
> > Yes, I'll check it with and without activation. 
> > 
> > I have another proposal for the changes that need to make it in for
> > 4.7
> > in a distinct thread.
> 
> I noticed that...

I've fixed a couple of issues and am tidying that up today.  I've
verified it allows a downgrade of a backend TDB by editing @INDEXLIST,
which could then be followed by a dbcheck --reindex.  

This is helpful if folks downgrade from 4.8 to 4.7 and forget the run
the (to be written) downgrade script first. 

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170907/daa59e79/signature.sig>