ccan dependencies seems to break external projects

Andrew Bartlett abartlet at samba.org
Sat Feb 18 16:01:04 MST 2012


On Sat, 2012-02-18 at 16:40 -0500, simo wrote:
> On Sat, 2012-02-18 at 11:08 +1100, Andrew Bartlett wrote: 
> > On Mon, 2012-02-13 at 08:04 -0500, Stephen Gallagher wrote:
> > > On Mon, 2012-02-13 at 08:08 +1100, Andrew Bartlett wrote:
> > > > On Sun, 2012-02-12 at 19:08 +0100, Jelmer Vernooij wrote:
> > > > > On 02/12/2012 06:57 PM, Matthieu Patou wrote:
> > > > > > On 02/12/2012 04:52 AM, Jelmer Vernooij wrote:
> > > > > >> On Sat, Feb 11, 2012 at 02:49:20PM -0800, Matthieu Patou wrote:
> > > > > >>> I'm trying to compile openchange as described here : 
> > > > > >>> http://tracker.openchange.org/projects/openchange/wiki/HowTo_Install_OpenChange_From_Source
> > > > > >>>
> > > > > >>> I'm using the latest git tree instead of alpha17, after a couple of
> > > > > >>> tweak in the script that setup the prerequisits I have a samba4 with
> > > > > >>> using system talloc/tdb/tevent and ldb.
> > > > > >>>
> > > > > >>> Then I try to build openchange and got:
> > > > > >>>
> > > > > >>>
> > > > > >>> /usr/local/src/openchange$ make
> > > > > >>> Linking sample application bin/libmapixx-test
> > > > > >>> g++ -I.  -O3 -Wall -g3 -fstrict-aliasing -Wp,-D_FORTIFY_SOURCE=2
> > > > > >>> -DHAVE_IMMEDIATE_STRUCTURES=1 -D_GNU_SOURCE=1
> > > > > >>> -Wl,-rpath,/usr/local/samba/lib -I/usr/local/samba/include
> > > > > >>>     -I/usr/local/samba/include   -I/usr/local/samba/include
> > > > > >>> -I/usr/local/samba/include     -o bin/libmapixx-test
> > > > > >>> libmapi++/tests/test.cpp libmapipp.so.0.11 libmapi.so.0
> > > > > >>> .11 -Wl,-rpath,/usr/local/samba/lib -L/usr/local/samba/lib -ldcerpc
> > > > > >>> -ldcerpc-binding -lndr -lsamba-hostconfig -lsamba-util -ltevent
> > > > > >>> -ltalloc   -Wl,-rpath,/usr/local/samb
> > > > > >>> a/lib -L/usr/local/samba/lib -lldb -ltalloc
> > > > > >>> -Wl,-rpath,/usr/local/samba/lib -L/usr/local/samba/lib -ltalloc
> > > > > >>> -lpthread
> > > > > >>> /usr/local/samba/lib/libsmbconf.so.0: undefined reference to
> > > > > >>> `hash_any at SAMBA_4.0.0ALPHA18_GIT_0B3A262'
> > > > > >>>
> > > > > >>>
> > > > > >>> The hash_any symbol comes from the ccan library and is included in
> > > > > >>> libsmbconf.so via source3/util/
> > > > > >>>
> > > > > >>> I suspect there is a symbols collision when you build samba 4 with
> > > > > >>> system libs as ldb has its own version of ccan but obviously with a
> > > > > >>> different version although it seems that ldb didn't directly need
> > > > > >>> ccan (no ccan in the output of ldd libldb.so), the following command
> > > > > >>> ls /usr/local/systemsamba/lib/ldb/libccan.so.0* output this:
> > > > > >>>
> > > > > >>> /usr/local/systemsamba/lib/ldb/libccan.so.0
> > > > > >>> /usr/local/systemsamba/lib/ldb/libccan.so.0.1-init-1161-g661d41f
> > > > > >>>
> > > > > >>> There is no way to have a system ccan and it seems that there is no
> > > > > >>> easy way to use the one built for ldb as the "system" ccan.
> > > > > >>>
> > > > > >>> I don't know what can be done, the cheapest is to have a
> > > > > >>> source3/util not to depend on ccan if --disable-tdb2 was specified
> > > > > >>> on the command line.
> > > > > >> Does ccan really have to be a separate library? Its goal seems to be
> > > > > >> the same as the 'samba-util' library.
> > > > > >>
> > > > > >> If I understand correctly ccan isn't really meant as a shared library
> > > > > >> with utility functions but rather as a collection of C files that you
> > > > > >> can pick a few useful ones from.
> > > > > >>
> > > > > >> Would it perhaps be possible to just include the relevant files from
> > > > > >> ccan in libsamba-util?
> > > > > > I'm not sure how this will fix the problem.
> > > > > > Let say that we have ccan in samba-util, instead of depending on ccan, 
> > > > > > ldb will depend on samba-util.
> > > > > > It means if I understand well that we will have a libsambautil.so with 
> > > > > > a version 1.4.1 (the version of ldb) but as it seems that we have no 
> > > > > > way to use a system samba-util library when building samba4 will have 
> > > > > > also a version of this library SAMBA_4.0.0ALPHA...
> > > > > tdb can include whatever it needs from ccan in the same way as 
> > > > > samba-util does (AFAIK it only uses a single macro from ccan at the moment).
> > > > > 
> > > > > ccan doesn't have a well-defined ABI. The problem with the way you're 
> > > > > building ldb and Samba is that you end up with two versions of the ccan 
> > > > > library - one which does have hash_any and one which doesn't. The linker 
> > > > > only tries to load one and not the other, because it remembers it has 
> > > > > already loaded ccan.
> > > > 
> > > > Does removing the 'vnum' line from ccan/wscript fix any of this?
> > > > 
> > > > It seems odd to be forcing the vnum from the upstream ccan if this
> > > > really is a private library?
> > > > 
> > > > Also, shouldn't we be putting a -samba4 prefix on the end of the name of
> > > > all the private and bundled libraries?  (We do it when bundling
> > > > Heimdal).
> > > > 
> > > > Andrew Bartlett
> > > > 
> > > 
> > > I don't think that encoding the version number in the library is a
> > > sufficient answer since, as discussed earlier in this thread, it is
> > > possible to build subsets of CCAN. This would presumably be the same
> > > library version but would be notably missing some components and
> > > therefore ABI-incompatible with an application linked against it.
> > > 
> > > There really only seem like two options:
> > > 1) CCAN needs to be modified so that we always build ALL components of
> > > it into a shared library. Ideally we would do this upstream with CCAN
> > > (and not privately in samba). Then we could have upstream release
> > > tarballs with a defined version that samba should then link against. I'm
> > > aware that Samba likes to have the ability to also build it in-tree, but
> > > that's a secondary concern. This option allows distributions to package
> > > a system library version of ccan (which is the preferred approach for
> > > most distributions, since it allows easy upgrades to fix bugs and
> > > security issues across all platforms).
> > > 
> > > 2) If CCAN truly cannot be built as a common library (like libreplace
> > > cannot), then the only correct answer should be to disallow ever
> > > building it as a shared object. We should allow only static linking (or
> > > pure copylib, like libreplace). If we're going purely copylib, we should
> > > remove the build-system entirely so that it can only be used by
> > > incorporating it into the Makefile/WAF build of its consumers. In other
> > > words, we should make it impossible to produce a system library or
> > > shared object that could ever conflict with the internal version.
> > > 
> > > I'm personally in favor of option 1 if it is at all possible.
> > 
> > 3) Install bundled libs in the private dir (as we currently do) with a
> > -samba4 suffix like we do with Heimdal.  I thought we used to do this
> > with all our bundled libs, but there has been a number of changes and
> > different approaches here over time.  This will also fix the popt
> > issue. 
> 
> This is not an acceptable option for many distributions. You need to
> jump thorugh hoops to justify why you want to use a private copy of a
> library instead of the version in the distribution, and for good
> reasons.

Well we can't do 2) either, as then we will end up with duplicate
symbols, which our build system correctly denies.  It would need to
remain a private Samba library, like all the others.

Rusty will have to indicate if he wishes to make ccan upstream a .so
with an ABI.  In the absence of that, I cannot see any alternative to
3).  This at least also helps other non-linux, non distributions systems
that may get popt multiple times as well. 

In the meantime, for the release I hope to make very shortly,
--disable-tdb2 seems the only option for distributions with these
(reasonable, just difficult for rusty's ccan modal) rules. 

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org



More information about the samba-technical mailing list