ccan dependencies seems to break external projects

[Stephen Gallagher]

On Mon, 2012-02-13 at 08:08 +1100, Andrew Bartlett wrote:
> On Sun, 2012-02-12 at 19:08 +0100, Jelmer Vernooij wrote:
> > On 02/12/2012 06:57 PM, Matthieu Patou wrote:
> > > On 02/12/2012 04:52 AM, Jelmer Vernooij wrote:
> > >> On Sat, Feb 11, 2012 at 02:49:20PM -0800, Matthieu Patou wrote:
> > >>> I'm trying to compile openchange as described here : 
> > >>> http://tracker.openchange.org/projects/openchange/wiki/HowTo_Install_OpenChange_From_Source
> > >>>
> > >>> I'm using the latest git tree instead of alpha17, after a couple of
> > >>> tweak in the script that setup the prerequisits I have a samba4 with
> > >>> using system talloc/tdb/tevent and ldb.
> > >>>
> > >>> Then I try to build openchange and got:
> > >>>
> > >>>
> > >>> /usr/local/src/openchange$ make
> > >>> Linking sample application bin/libmapixx-test
> > >>> g++ -I.  -O3 -Wall -g3 -fstrict-aliasing -Wp,-D_FORTIFY_SOURCE=2
> > >>> -DHAVE_IMMEDIATE_STRUCTURES=1 -D_GNU_SOURCE=1
> > >>> -Wl,-rpath,/usr/local/samba/lib -I/usr/local/samba/include
> > >>>     -I/usr/local/samba/include   -I/usr/local/samba/include
> > >>> -I/usr/local/samba/include     -o bin/libmapixx-test
> > >>> libmapi++/tests/test.cpp libmapipp.so.0.11 libmapi.so.0
> > >>> .11 -Wl,-rpath,/usr/local/samba/lib -L/usr/local/samba/lib -ldcerpc
> > >>> -ldcerpc-binding -lndr -lsamba-hostconfig -lsamba-util -ltevent
> > >>> -ltalloc   -Wl,-rpath,/usr/local/samb
> > >>> a/lib -L/usr/local/samba/lib -lldb -ltalloc
> > >>> -Wl,-rpath,/usr/local/samba/lib -L/usr/local/samba/lib -ltalloc
> > >>> -lpthread
> > >>> /usr/local/samba/lib/libsmbconf.so.0: undefined reference to
> > >>> `hash_any at SAMBA_4.0.0ALPHA18_GIT_0B3A262'
> > >>>
> > >>>
> > >>> The hash_any symbol comes from the ccan library and is included in
> > >>> libsmbconf.so via source3/util/
> > >>>
> > >>> I suspect there is a symbols collision when you build samba 4 with
> > >>> system libs as ldb has its own version of ccan but obviously with a
> > >>> different version although it seems that ldb didn't directly need
> > >>> ccan (no ccan in the output of ldd libldb.so), the following command
> > >>> ls /usr/local/systemsamba/lib/ldb/libccan.so.0* output this:
> > >>>
> > >>> /usr/local/systemsamba/lib/ldb/libccan.so.0
> > >>> /usr/local/systemsamba/lib/ldb/libccan.so.0.1-init-1161-g661d41f
> > >>>
> > >>> There is no way to have a system ccan and it seems that there is no
> > >>> easy way to use the one built for ldb as the "system" ccan.
> > >>>
> > >>> I don't know what can be done, the cheapest is to have a
> > >>> source3/util not to depend on ccan if --disable-tdb2 was specified
> > >>> on the command line.
> > >> Does ccan really have to be a separate library? Its goal seems to be
> > >> the same as the 'samba-util' library.
> > >>
> > >> If I understand correctly ccan isn't really meant as a shared library
> > >> with utility functions but rather as a collection of C files that you
> > >> can pick a few useful ones from.
> > >>
> > >> Would it perhaps be possible to just include the relevant files from
> > >> ccan in libsamba-util?
> > > I'm not sure how this will fix the problem.
> > > Let say that we have ccan in samba-util, instead of depending on ccan, 
> > > ldb will depend on samba-util.
> > > It means if I understand well that we will have a libsambautil.so with 
> > > a version 1.4.1 (the version of ldb) but as it seems that we have no 
> > > way to use a system samba-util library when building samba4 will have 
> > > also a version of this library SAMBA_4.0.0ALPHA...
> > tdb can include whatever it needs from ccan in the same way as 
> > samba-util does (AFAIK it only uses a single macro from ccan at the moment).
> > 
> > ccan doesn't have a well-defined ABI. The problem with the way you're 
> > building ldb and Samba is that you end up with two versions of the ccan 
> > library - one which does have hash_any and one which doesn't. The linker 
> > only tries to load one and not the other, because it remembers it has 
> > already loaded ccan.
> 
> Does removing the 'vnum' line from ccan/wscript fix any of this?
> 
> It seems odd to be forcing the vnum from the upstream ccan if this
> really is a private library?
> 
> Also, shouldn't we be putting a -samba4 prefix on the end of the name of
> all the private and bundled libraries?  (We do it when bundling
> Heimdal).
> 
> Andrew Bartlett
> 

I don't think that encoding the version number in the library is a
sufficient answer since, as discussed earlier in this thread, it is
possible to build subsets of CCAN. This would presumably be the same
library version but would be notably missing some components and
therefore ABI-incompatible with an application linked against it.

There really only seem like two options:
1) CCAN needs to be modified so that we always build ALL components of
it into a shared library. Ideally we would do this upstream with CCAN
(and not privately in samba). Then we could have upstream release
tarballs with a defined version that samba should then link against. I'm
aware that Samba likes to have the ability to also build it in-tree, but
that's a secondary concern. This option allows distributions to package
a system library version of ccan (which is the preferred approach for
most distributions, since it allows easy upgrades to fix bugs and
security issues across all platforms).

2) If CCAN truly cannot be built as a common library (like libreplace
cannot), then the only correct answer should be to disallow ever
building it as a shared object. We should allow only static linking (or
pure copylib, like libreplace). If we're going purely copylib, we should
remove the build-system entirely so that it can only be used by
incorporating it into the Makefile/WAF build of its consumers. In other
words, we should make it impossible to produce a system library or
shared object that could ever conflict with the internal version.

I'm personally in favor of option 1 if it is at all possible.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120213/4f3ca4bd/attachment.pgp>