[PATCH] LMDB full patch set

[Andrew Bartlett]

On Sat, 2018-05-19 at 07:14 +1200, Andrew Bartlett via samba-technical
wrote:
> On Fri, 2018-05-18 at 13:13 +0200, Stefan Metzmacher wrote:
> > Am 18.05.2018 um 10:34 schrieb Andrew Bartlett via samba-technical:
> > > On Thu, 2018-05-17 at 16:15 +1200, Garming Sam wrote:
> > > > I suspect Joe's concurrent stress-testing would tell us more about how
> > > > it behaves under real load -- we definitely should block less now,
> > > > although not as little as LMDB fully allows due to other requirements.
> > > 
> > > Just to follow up on this with some numbers from an internal thread.  
> > > 
> > > We got a load of 167 (I think operations / second) on lmdb, essentially
> > > the same as the 160 we saw on master.  Windows 2012R2 gives 235 on the
> > > same virtual hardware, for reference. 
> > > 
> > > So no massive gain, but no massive loss either.  This will likely
> > > change with larger user sets (I don't have the details to hand as to
> > > what the DB size was).
> > 
> > Thanks! I also compared autobuild times and got similar results.
> > 
> > One thing I was wondering about is if we should make use of virtual
> > databases within lmdb. 
> > I'm not asking to do that before we can push this, but we should
> > at least have ways for a seamless change in future. The compatibility
> > flags at the samba_dsdb level are not a valid way to detect this!
> > We need this at the ldb_lmdb layer.
> 
> Yes, we have had many interesting ideas about ways we can properly
> leverage the virtual databases.  We certainly could solve a lot of
> problems that way, and (for example) if we use a sub-db per index we
> gain the ability to do >= searches but walking the keys in-order.
> 
> I agree, we shouldn't block on this.  Attached and included in the
> branch is a simple patch to recognise this probable future and abort on
> downgrade.

BTW, the pipeline for this updated branch is:
https://gitlab.com/catalyst-samba/samba/pipelines/22279826

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba