[PATCH] WIP: indexing for <= and >=
aaronhaslett at catalyst.net.nz
Thu Mar 28 23:59:26 UTC 2019
Garming is right, my mistake. Please ignore the join timing changes.
Only the manual replication timing is relevant.
Here's version 3 of <= and >= indexing patch. This version cleans up
the patches to submission standard, and increases the test coverage.
On 21/03/19 11:03 AM, Garming Sam wrote:
> On 20/03/19 5:30 PM, Aaron Haslett via samba-technical wrote:
>> Here's version 2 of <= and >= indexing patch:
>> * bug fixes
>> * bumping index version to trigger reindex
>> * new LDB version due to adding index_format_fn field to schema syntax
>> Testing was done on a 300,000 record DC with and without the patch to
>> find what speedup the patch achieved:
>> * Join time reduced from 3186s to 1446s
> This improvement is using the changes to the in-memory tdb size that
> Gary is talking about in "Re-index performance - default tdb sizes." --
> but that is a pretty noticeable improvement. We need to do some more
> testing of just this patch (or Gary's), and also what effect this has on
> ongoing full synchronizations.
>> * Manual replication call with no data replicated reduced from 4s to 1s
>> * Manual replication call with 200 record replicated reduced from 6.5s
>> to 2s
> To improve the performance from here, there likely needs to be more work
> done on the rest of the replication call chain. (Around 0.2 seconds of
> the call is in the actual GUID scan search, although there are still
> some ways we can improve this time too by avoiding allocations of DNs).
>> On 19/03/19 2:12 PM, Aaron Haslett via samba-technical wrote:
>>> Sorry, I forgot to say why we're doing this. We want to be able to
>>> greatly speed up "uSNChanged >= x" queries which are used during
>>> replication. Testing on a DC with 300,000 records showed a speed up
>>> from ~4s to ~0.2s. The patch only supports integers on LMDB for now.
>>> On 19/03/19 12:42 PM, Aaron Haslett via samba-technical wrote:
>>>> Work in progress implementation of <= and >= indexing. Adds
>>>> iterate_range to LDB API and uses it for indexing. Modifications to come
>>>> because right now this patch modifies a core LDB type which will
>>>> probably break things. I'll change it soon to add a new ordered integer
>>>> type with a new OID.
>>>> CI: https://gitlab.com/catalyst-samba/samba/pipelines/52460313
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 71366 bytes
Desc: not available
More information about the samba-technical