[PATCH] fix various build problems

[Michael Adam]

I just noticed that the patch posted yesterday
was incomplete. Updated patchset attached.

Michael

On 2014-10-28 at 18:28 +0100, Michael Adam wrote:
> Hi,
> 
> We have a certain group of problems with our build:
> 
> These show up when certain FOO-config programs
> (like cups-config or python-config) report library
> paths for ldflags (-L/some/path) which are actually
> standard lib paths for the system. pkg-config
> does NOT do that by default, but the above two do.
> 
> Then what happens is that in the linking phase,
> a too old version of a library which is installed
> in that standard libpath is chosen instead of the
> newer one that was reported by pkg-config.
> 
> A typical example is a new version of tdb, that
> e.g. supports robust mutexs. A test is running
> 
> ./script/autobuild.py samba-ctdb on a system
> that supports robust mutexes. This test first
> installs tdb into the prefix and then
> configures and builds samba against this.
> Linking fails.
> 
> Another example is bug
> https://bugzilla.samba.org/show_bug.cgi?id=10515
> which prevents builds on FreeBSD to succeed under
> certain circumstances.
> 
> Attached is a patch to fix this problem.
> It does so by filtering out the standard directories.
> It uses ${CC} -print-search-dirs to build
> the list of standard lib dirs. At least gcc and clang
> support this.
> 
> The patch may need some more changes, because it
> currently touches wafadmin to filter the standard paths.
> 
> Doing it in wafsamba would be more complicated because
> one would need to parse and dissect the LINKFLAGS
> again. The wafadmin code can act on the pure paths.
> And I'd need to find a proper place to do it.
> 
> But I wanted to share this patch for testing
> and feedback now that I have really spent a lot of
> time on it. A first hack that was system-specific
> dates back to June and was done together with Metze,
> who also showed -print-search-dirs to me ...
> 
> Comments please! :-)
> 
> Thanks, Michael

> From 6a8df20f9c8cc91d553c527ace5bf5ba01290519 Mon Sep 17 00:00:00 2001
> From: Michael Adam <obnox at samba.org>
> Date: Wed, 11 Jun 2014 15:49:33 +0200
> Subject: [PATCH] TODO: build: fix issue with standard libpaths introduced by
>  cups-config and friends
> 
> Problem:
> 
> Some tools like cups-config and python-config list system standard library
> paths with -L in the ldflags.
> 
> Since these flags are in treated as flags required by libraries
> and not as standard flags, this can lead to situations where
> configure correctly detects a version of a library (e.g. talloc)
> as installed into a special location, but the link process fails
> because the unsupported old version of the library is found first
> in the library path reported by foo-config.
> 
> See e.g.:
> 
> BUG: https://bugzilla.samba.org/show_bug.cgi?id=10515
> 
> Solution:
> 
> The solution of this patch is to filter out the standard
> library paths. The difficulty here is that the list of standard
> paths varies between the systems. This patch calls "${CC} -print-search-dirs"
> to get this list, which is supported at least by gcc and clang.
> 
> TODOS:
> - is_standard_libpath() : correct place in ccroot.py ??
> - can/should we do it in wafsamba only?
> ---
>  buildtools/wafadmin/Tools/ccroot.py   | 10 ++++++++++
>  buildtools/wafadmin/Tools/config_c.py | 10 +++++++---
>  buildtools/wafadmin/Tools/python.py   |  4 +++-
>  buildtools/wafsamba/wscript           | 29 +++++++++++++++++++++++++++++
>  4 files changed, 49 insertions(+), 4 deletions(-)
> 
> diff --git a/buildtools/wafadmin/Tools/ccroot.py b/buildtools/wafadmin/Tools/ccroot.py
> index 264bdc7..e68961b 100644
> --- a/buildtools/wafadmin/Tools/ccroot.py
> +++ b/buildtools/wafadmin/Tools/ccroot.py
> @@ -20,6 +20,12 @@ import config_c # <- necessary for the configuration, do not touch
>  
>  USE_TOP_LEVEL = False
>  
> +def is_standard_libpath(env, path):
> +	for _path in env.STANDARD_LIBPATH:
> +		if _path == os.path.normpath(path):
> +			return True
> +	return False
> +
>  def get_cc_version(conf, cc, gcc=False, icc=False):
>  
>  	cmd = cc + ['-dM', '-E', '-']
> @@ -467,10 +473,14 @@ def apply_obj_vars(self):
>  		v.append_value('LINKFLAGS', v['FULLSTATIC_MARKER'])
>  
>  	for i in v['RPATH']:
> +		if is_standard_libpath(v, i):
> +			continue
>  		if i and rpath_st:
>  			app('LINKFLAGS', rpath_st % i)
>  
>  	for i in v['LIBPATH']:
> +		if is_standard_libpath(v, i):
> +			continue
>  		app('LINKFLAGS', libpath_st % i)
>  		app('LINKFLAGS', staticlibpath_st % i)
>  
> diff --git a/buildtools/wafadmin/Tools/config_c.py b/buildtools/wafadmin/Tools/config_c.py
> index d0bc617..2c8a6c7 100644
> --- a/buildtools/wafadmin/Tools/config_c.py
> +++ b/buildtools/wafadmin/Tools/config_c.py
> @@ -11,6 +11,7 @@ from Utils import md5
>  import Build, Utils, Configure, Task, Options, Logs, TaskGen
>  from Constants import *
>  from Configure import conf, conftest
> +import ccroot
>  
>  cfg_ver = {
>  	'atleast-version': '>=',
> @@ -60,7 +61,8 @@ def parse_flags(line, uselib, env):
>  			app('LIB_' + uselib, ot)
>  		elif st == '-L':
>  			if not ot: ot = lst.pop(0)
> -			app('LIBPATH_' + uselib, ot)
> +			if not ccroot.is_standard_libpath(env, ot):
> +				app('LIBPATH_' + uselib, ot)
>  		elif x == '-pthread' or x.startswith('+'):
>  			app('CCFLAGS_' + uselib, x)
>  			app('CXXFLAGS_' + uselib, x)
> @@ -83,9 +85,11 @@ def parse_flags(line, uselib, env):
>  		# in too old versions of our internal libs.
>  		#
>  		elif x.startswith('-Wl,-R'):
> -			app('RPATH_' + uselib, x[6:])
> +			if not is_wellknown_path(x[6:]):
> +				app('RPATH_' + uselib, x[6:])
>  		elif x.startswith('-Wl,-rpath,'):
> -			app('RPATH_' + uselib, x[11:])
> +			if not is_wellknown_path(x[11:]):
> +				app('RPATH_' + uselib, x[11:])
>  		elif x.startswith('-Wl'):
>  			app('LINKFLAGS_' + uselib, x)
>  		elif x.startswith('-m') or x.startswith('-f'):
> diff --git a/buildtools/wafadmin/Tools/python.py b/buildtools/wafadmin/Tools/python.py
> index 35c61c2..d5ee841 100644
> --- a/buildtools/wafadmin/Tools/python.py
> +++ b/buildtools/wafadmin/Tools/python.py
> @@ -11,6 +11,7 @@ from Logs import debug, warn, info
>  from TaskGen import extension, before, after, feature
>  from Configure import conf
>  from config_c import parse_flags
> +import ccroot
>  
>  EXT_PY = ['.py']
>  FRAG_2 = '''
> @@ -249,7 +250,8 @@ MACOSX_DEPLOYMENT_TARGET = %r
>  		result = conf.check(lib=name, uselib='PYEMBED', libpath=path)
>  
>  	if result:
> -		env['LIBPATH_PYEMBED'] = path
> +		if not ccroot.is_standard_libpath(env, path[0]):
> +			env['LIBPATH_PYEMBED'] = path
>  		env.append_value('LIB_PYEMBED', name)
>  	else:
>  		conf.log.write("\n\n### LIB NOT FOUND\n")
> diff --git a/buildtools/wafsamba/wscript b/buildtools/wafsamba/wscript
> index 1a2cfe6..4faf768 100755
> --- a/buildtools/wafsamba/wscript
> +++ b/buildtools/wafsamba/wscript
> @@ -192,6 +192,33 @@ def set_options(opt):
>                     help='tag release in git at the same time',
>                     type='string', action='store', dest='TAG_RELEASE')
>  
> +#
> +# Determine the standard libpath for the used compiler,
> +# so we can later use that to filter out these standard
> +# library paths when some tools like cups-config or
> +# python-config report standard lib paths with their
> +# ldflags (-L...)
> +#
> +def get_cc_std_libpath(cc):
> +    # at least gcc and clang support this:
> +    try:
> +        cmd = cc + ['-print-search-dirs']
> +        out = Utils.cmd_output(cmd).split('\n')
> +    except ValueError:
> +        # option not supported by compiler - use a standard list of directories
> +        dirlist = [ '/usr/lib', '/usr/lib64' ]
> +    except:
> +        raise Utils.WafError('Unexpected error running "%s"' % (cmd,))
> +    else:
> +        dirlist = []
> +        for line in out:
> +            line = line.strip()
> +            if line.startswith("libraries: ="):
> +                dirliststr = line[len("libraries: ="):]
> +                dirlist = [ os.path.normpath(x) for x in dirliststr.split(':') ]
> +                break
> +
> +    return dirlist
>  
>  @wafsamba.runonce
>  def configure(conf):
> @@ -211,6 +238,8 @@ def configure(conf):
>  
>      conf.check_tool('compiler_cc')
>  
> +    conf.env.STANDARD_LIBPATH = get_cc_std_libpath(conf.env.CC)
> +
>      # we need git for 'waf dist'
>      conf.find_program('git', var='GIT')
>  
> -- 
> 1.9.1
> 



-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-TODO-build-fix-issue-with-standard-libpaths-introduc.patch
Type: text/x-diff
Size: 6092 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141029/e854aff0/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141029/e854aff0/attachment.pgp>