(again) Ship ldb like libndr and ldbwbclient? As a Samba public library!

[Andrew Bartlett]

I've asked this a few times before, and I'm hesitant to ask again, but
I feel quite strongly on this and want to try one last time to make
this easier for our release management and packaging. 

I really thing that, while ldb had noble aims to be as popular as tdb
and talloc, that ldb should be integrated back into Samba as a Samba
public library, but not an independent release. 

Rationale
=========

I ask this for the following reasons:

Samba-enforced lockstep
-----------------------

Fundamentally, ldb is generously provided to the world, and used in
sssd in particular, but is developed by and for Samba.  I'm aware that
there were early contributions with broader goals, but without wanting
to hurt any feelings, this this a Samba effort these days.

Samba strictly requires the version included in the Samba release.  Any
other version is untested, and Samba uses an unstable ABI (the modules
ABI) and relies on specific behaviours (we fix bugs and rely on those
fixes). 

There is significant build rule complexity to enforce this requirement,
yet for some reason this isn't as strict as our runtime rule (release
version can float at build time, but not at runtime)!

LDB Selftest
------------

Because ldb does not use the full Samba selftest system, we can't do
knownfail on tests.  This means our standard practice of having a
failing test first in the commit stream, with a knownfail entry,
followed by the revert when the fix can't work.

While of course I trust my fellow developers to always test that their
tests show the bug, I've certainly found our main-Samba pattern helpful
in ensuring this is the case, as 'make test TESTS=foo' can be run on
each commit.

Samba Selftest
--------------

We also introduce untested (in autobuild) complexity to the Samba
selftest system, eg 
python:tests: Fix running tests with system libldb
https://gitlab.com/samba-team/samba/-/merge_requests/2026

I'll merge that patch, but I wish we didn't need this.

(security) Release time
-----------------------

The need to, at the last moment, make a ldb release (because someone
like myself has forgotten to include that in the patches) is a extra
stress on our release management.  While ABI versions should be bumped,
a missing ABI bump is not as catastrophic as a whole missing release.

The same applies to our distributors, who for every LDB security
release need to ship ldb, but also deal with the fact that Samba is
strictly building against the bundled ldb version.  This brings
additional complexity and makes it harder to track LDB CVEs, as they
are issued against Samba bug apply to ldb.

see eg
https://security-tracker.debian.org/tracker/CVE-2019-3824


My proposal
===========

LDB would remain LGPL, but would be built akin to libwbclient,
libsmbclient, libndr and other things that broader system packages rely
on.  We would not search for or use a 'system' ldb, we would only use
the one we shipped with.

A user wishing only ldb can install the ldb package from a distributor,
who can continue to package the ldb libraries and binaries separately,
just as they can install libndr for sssd use via samba-common. 

The independent ldb build would be removed to reduce the complexity for
developers.  A user building from source wishing to build only ldb
would build Samba - spending some CPU time - and select to remove other
than libldb and the modules if they really want.  Samba is developer-
resource constrained, so we need to make choices to maintain simplicity
in our project.


Next steps
==========

I recognise this has been a controversial issue in the past, so as to
be effort-efficient I'm not going wait for a seconder for the proposal
to ensure I'm on the right track, then make a MR.

Likewise if you have significant concerns from real-world use cases
please see if we can work something out, but avoid 'straw-man' examples
please:  LDB is over a decade old, if a use case is needed it would
have come up by now.

Conclusion
==========

LDB is used outside Samba, by sssd, but has not flourished in the
current sate as a semi-independent project, is very unlikely to become
fully independent and should now be fully folded back into main-Samba. 

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett (he/him)       https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead, Catalyst IT   https://catalyst.net.nz/services/samba

Samba Development and Support, Catalyst IT - Expert Open Source
Solutions