ccan dependencies seems to break external projects

[Stephen Gallagher]

On Wed, 2012-02-22 at 08:41 +1030, Rusty Russell wrote:
> On Mon, 13 Feb 2012 08:04:14 -0500, Stephen Gallagher <sgallagh at redhat.com> wrote:
> > I don't think that encoding the version number in the library is a
> > sufficient answer since, as discussed earlier in this thread, it is
> > possible to build subsets of CCAN. This would presumably be the same
> > library version but would be notably missing some components and
> > therefore ABI-incompatible with an application linked against it.
> 
> Hi Stephen,
> 
>         Thanks for bringing these issues up.  It's important to figure
> out how to solve this not just for SAMBA, but for advice to other
> complex CCAN-using projects in future.
> 
> > There really only seem like two options:
> > 1) CCAN needs to be modified so that we always build ALL components of
> > it into a shared library. Ideally we would do this upstream with CCAN
> > (and not privately in samba). Then we could have upstream release
> > tarballs with a defined version that samba should then link against. I'm
> > aware that Samba likes to have the ability to also build it in-tree, but
> > that's a secondary concern. This option allows distributions to package
> > a system library version of ccan (which is the preferred approach for
> > most distributions, since it allows easy upgrades to fix bugs and
> > security issues across all platforms).
> 
> Making CCAN a versioned library is messy.  Upstream, CCAN is simply a
> collection of code snippets with no set API, let alone ABI.  That
> decision is up to individual module authors.
> 
> This means SAMBA would be bumping the SO major version on every change
> we import, or make the fork permanent, which would be painful and make
> it harder to grab future CCAN modules.
> 

See below, but I think the answer here is to build individual shared
objects per module, rather than a single one for all of CCAN.

> > 2) If CCAN truly cannot be built as a common library (like libreplace
> > cannot), then the only correct answer should be to disallow ever
> > building it as a shared object. We should allow only static linking (or
> > pure copylib, like libreplace). If we're going purely copylib, we should
> > remove the build-system entirely so that it can only be used by
> > incorporating it into the Makefile/WAF build of its consumers. In other
> > words, we should make it impossible to produce a system library or
> > shared object that could ever conflict with the internal version.
> > 
> > I'm personally in favor of option 1 if it is at all possible.
> 
> I don't know how to do this :( (Note that as of yesterday, there are
> significant changes to the tdb2 build, and it's now a public library.  I
> don't know if that changes anything).
> 
> If the problem is that (unlike samba-util) it's used by public libraries
> (ie. tdb2), then we can wean tdb2 off CCAN, copying in the bits we need.
> It's ugly, but has the benefit of making tdb2 entirely self-contained.
> 

Yeah, that's the crux of the issue. If I have a project that links
against some features of CCAN (or links against a system copy of the
library, if someone opts to maintain it this way for a distribution) and
I also attempt to link against libtdb, I run the risk of ending up with
conflicting symbols.

Obviously, the best answer (in my estimation) would be to come up with a
a way to produce system libraries that offer the CCAN features and can
be versioned sanely.


> Or we can go part way, and librify some CCAN modules; ccan/hash has had
> a particularly stable API, for example, and it's the largest one used by
> tdb2.  For this, I'd want to add checks to ccanlint to try to ensure ABI
> stability so it doesn't accidentally get broken, but it's possible.

It sounds to me like CCAN is less of a library and more of a collection
of libraries. So in terms of packaging, it might make sense to package
each module as its own shared library such as libccan_hash. Then we can
maintain the module versions individually and handle SOname bumps when
they're necessary for individual modules.

Whether to package these modules as separate package distributions (in
RPM parlance, as separate SRPMs) or just as subpackages built from one
set of common sources would be up to the individual distribution
packagers. As long as we can guarantee that libccan_hash.so.0 is
ABI-compatible (or bumped if it ceases to be), it becomes manageable to
maintain a system copy of these libraries.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120221/31725856/attachment.pgp>