(again) Ship ldb like libndr and ldbwbclient? As a Samba public library!

[Andrew Bartlett]

Any thoughts on this?  The idea is to make our release and security
release management easier by integrating one of our moving parts.

Andrew Bartlett

On Mon, 2021-06-21 at 09:57 +1200, Andrew Bartlett via samba-technical
wrote:
> I've asked this a few times before, and I'm hesitant to ask again, but
> I feel quite strongly on this and want to try one last time to make
> this easier for our release management and packaging. 
> 
> I really thing that, while ldb had noble aims to be as popular as tdb
> and talloc, that ldb should be integrated back into Samba as a Samba
> public library, but not an independent release. 
> 
> Rationale
> =========
> 
> I ask this for the following reasons:
> 
> Samba-enforced lockstep
> -----------------------
> 
> Fundamentally, ldb is generously provided to the world, and used in
> sssd in particular, but is developed by and for Samba.  I'm aware that
> there were early contributions with broader goals, but without wanting
> to hurt any feelings, this this a Samba effort these days.
> 
> Samba strictly requires the version included in the Samba release.  Any
> other version is untested, and Samba uses an unstable ABI (the modules
> ABI) and relies on specific behaviours (we fix bugs and rely on those
> fixes). 
> 
> There is significant build rule complexity to enforce this requirement,
> yet for some reason this isn't as strict as our runtime rule (release
> version can float at build time, but not at runtime)!
> 
> LDB Selftest
> ------------
> 
> Because ldb does not use the full Samba selftest system, we can't do
> knownfail on tests.  This means our standard practice of having a
> failing test first in the commit stream, with a knownfail entry,
> followed by the revert when the fix can't work.
> 
> While of course I trust my fellow developers to always test that their
> tests show the bug, I've certainly found our main-Samba pattern helpful
> in ensuring this is the case, as 'make test TESTS=foo' can be run on
> each commit.
> 
> Samba Selftest
> --------------
> 
> We also introduce untested (in autobuild) complexity to the Samba
> selftest system, eg 
> python:tests: Fix running tests with system libldb
> https://gitlab.com/samba-team/samba/-/merge_requests/2026
> 
> I'll merge that patch, but I wish we didn't need this.
> 
> (security) Release time
> -----------------------
> 
> The need to, at the last moment, make a ldb release (because someone
> like myself has forgotten to include that in the patches) is a extra
> stress on our release management.  While ABI versions should be bumped,
> a missing ABI bump is not as catastrophic as a whole missing release.
> 
> The same applies to our distributors, who for every LDB security
> release need to ship ldb, but also deal with the fact that Samba is
> strictly building against the bundled ldb version.  This brings
> additional complexity and makes it harder to track LDB CVEs, as they
> are issued against Samba bug apply to ldb.
> 
> see eg
> https://security-tracker.debian.org/tracker/CVE-2019-3824
> 
> 
> My proposal
> ===========
> 
> LDB would remain LGPL, but would be built akin to libwbclient,
> libsmbclient, libndr and other things that broader system packages rely
> on.  We would not search for or use a 'system' ldb, we would only use
> the one we shipped with.
> 
> A user wishing only ldb can install the ldb package from a distributor,
> who can continue to package the ldb libraries and binaries separately,
> just as they can install libndr for sssd use via samba-common. 
> 
> The independent ldb build would be removed to reduce the complexity for
> developers.  A user building from source wishing to build only ldb
> would build Samba - spending some CPU time - and select to remove other
> than libldb and the modules if they really want.  Samba is developer-
> resource constrained, so we need to make choices to maintain simplicity
> in our project.
> 
> 
> Next steps
> ==========
> 
> I recognise this has been a controversial issue in the past, so as to
> be effort-efficient I'm not going wait for a seconder for the proposal
> to ensure I'm on the right track, then make a MR.
> 
> Likewise if you have significant concerns from real-world use cases
> please see if we can work something out, but avoid 'straw-man' examples
> please:  LDB is over a decade old, if a use case is needed it would
> have come up by now.
> 
> Conclusion
> ==========
> 
> LDB is used outside Samba, by sssd, but has not flourished in the
> current sate as a semi-independent project, is very unlikely to become
> fully independent and should now be fully folded back into main-Samba. 
> 
> Thanks,
> 
> Andrew Bartlett

-- 
Andrew Bartlett (he/him)        https://samba.org/~abartlet/
Samba Team Member (since 2001)  https://samba.org
Samba Developer, Catalyst IT    https://catalyst.net.nz/services/samba