[PATCH] LMDB full patch set

[Andrew Bartlett]

On Mon, 2018-05-07 at 17:57 +0200, Stefan Metzmacher wrote:
> Am 07.05.2018 um 05:02 schrieb Andrew Bartlett via samba-technical:
> > 
> > Metze,
> > 
> > I've addressed your concerns as far as practical[1].  If you could
> > please allow this to proceed into master I would most appreciate it. 
> 
> Can you move all tdb related patch to the start of the patchset,
> they should work on their own. Also the fork protection tests.
> 
> I don't see where ldb_tdb_test is used in tests...

I've fixed all this and pushed those to autobuild with Garming's
review.

> From reading the patches it seems that it will no longer possible
> to build ldb or an AD DC without lmdb, e.g. 32bit support will be gone.
> I don't think that's acceptable right now, maybe in a few years.

Yes, you are correct we will require lmdb for the AD DC and ldb
standlone builds.

lmdb is shipped on 32 bit ubuntu.  I've set up a test environment and
will confirm the LDB tests pass on it.  Howard Chu indicates earlier in
the thread that it is fully functional, but limited in DB size to 2GB.
 
The DB size issue is also the first issue I've found empirically, as we
got this while trying to set the default DB size:

[1295/4337] Compiling lib/ldb/ldb_mdb/ldb_mdb.c
../lib/ldb/ldb_mdb/ldb_mdb.c: In function lmdb_open_env:
../lib/ldb/ldb_mdb/ldb_mdb.c:726:2: error: large integer implicitly
truncated to unsigned type [-Werror=overflow]
  ret = mdb_env_set_mapsize(*env, 16LL * GIGABYTE);

As the operational side is both optional and not a major departure from
the 4GB limit on TDB, I think this is a reasonable restriction.  Those
running a RaspberryPi DC won't be running hundreds of users in any
case.

To address the broader point:

My strong preference is that the AD DC be able to rely on this feature
being available.  I don't mind if the fileserver-with-bundled-libraries 
has it optional, but for the AD DC I'm actively trying to reduce the
possible combinations so we can keep everything tested.  (Once this
lands, I'll change some of our selftest environments to use lmdb). 

That is why 'samba' needs to require it for the AD DC.  For standlone
ldb, that reason I've not made it optional is that otherwise we need to
declare to samba that the built-standalone 'system' ldb doesn't have
lmdb, then have a reliable method to detect that.  This seemed like all
to much complexity for a feature we will likely soon strictly rely on. 

(We have been actively removing the ability to run 'make test' without
optional packages after we found parts of our testsuite silently
skipped.  I'll propose the same for the ldb package once this is
agreed.)

Portability to i386 I can sort out, but can you confirm you are
otherwise happy with the branch?

https://gitlab.com/catalyst-samba/samba/commits/metze-master4-lmdb-full

Gary,

Can you do the same?

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180509/724608d7/signature.sig>