[PATCH] LMDB full patch set

[Stefan Metzmacher]

Am 22.05.2018 um 09:42 schrieb Andrew Bartlett via samba-technical:
> On Sat, 2018-05-19 at 07:14 +1200, Andrew Bartlett via samba-technical
> wrote:
>> On Fri, 2018-05-18 at 13:13 +0200, Stefan Metzmacher wrote:
>>>
>>> Thanks! I also compared autobuild times and got similar results.
>>>
>>> One thing I was wondering about is if we should make use of virtual
>>> databases within lmdb. 
>>> I'm not asking to do that before we can push this, but we should
>>> at least have ways for a seamless change in future. The compatibility
>>> flags at the samba_dsdb level are not a valid way to detect this!
>>> We need this at the ldb_lmdb layer.
> 
>> Yes, we have had many interesting ideas about ways we can properly
>> leverage the virtual databases.  We certainly could solve a lot of
>> problems that way, and (for example) if we use a sub-db per index we
>> gain the ability to do >= searches but walking the keys in-order.
>>
>> I agree, we shouldn't block on this.  Attached and included in the
>> branch is a simple patch to recognise this probable future and abort on
>> downgrade.
> 
> G'Day,
> 
> Having addressed all the various issues raised over the past weeks and
> months, I would now like to push this to master.  
> 
> There will always be things to improve, but we will learn those best
> when we have it available for use. 

Yes. One thing I tought about is starting with a small
map_size and handle MDB_MAP_FULL and MDB_MAP_RESIZED.
We could change the size in steps similar to tdb.
That would remove the 32bit problem and avoids huge number in
the 'top' output.

> I would like to push this in the next day or so.  

I've removed some leading tabs/whitespaces
and improved the commit message for ldb 1.4.0, see
https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/master4-lmdb-full

Should we mark lmdb as experimental in the release commit message?

I also noticed that autobuilds with mdb are a little bit slower.
I compared
https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=9b73105d902f
https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=95cfac07dc36
https://git.samba.org/?p=metze/samba/wip.git;a=commitdiff;h=0fd1ca4b8b33

"autobuild-private.sh samba" on the same host (at the same time)
with mdb we use about 3 mins more of about 160 mins.
and the same with "autobuild-private samba-ad-dc" with less difference.
But in all cases autobuild finished earlier when not using mdb.

metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180522/4c40bd73/signature.sig>