Serious problem with standard library paths and order of -L flags

[Michael Adam]

Hi list,

we currently have a problem with our waf build and library paths
handed in by some pkg-info or cups-info calls.

Especially cups-info seems to also report library search paths
which are standard paths of the system.

This leadas to problems when we have a e.g. libtdb installed
in a special directory, and an older version in the system
as happens in our autobuild (e.g. the clustered target samba-ctdb).

Then pkg-info for tdb works out nicely but the -Lfoo flag ends
up in the LDFLAGS after the -L/standard/path from (e.g.) cups.

So the configure and compile works (because with -I the
corresponding problem does not exist), but linking fails
because the build links in the too old library from the standard
path an symbols are missing.

As said, I observed this with autobuild and testing with new
features of tdb, but this already exists in real life, e.g.
for FreeBSD:
https://bugzilla.samba.org/show_bug.cgi?id=10515
but it is not FreeBSD specific.

I have two patches hacked into our waf build (pair-hacked
with Metze) that makes the internals skip a library path
handed in from (e.g.) pkg-config if it is a standard or
well-known library path. This fixes it for me on my
ubuntu 64 bit system, but of course this is not portable!
(patches attached...)

==> Question to the list (Jelmer?...):

Is there a good way to retrieve a list of "standard"
library paths from the system (via python)?

(Or does anybody have an idea how to solve this differntly?)

Any hints would be highly appreciated!

Cheers - Michael

-------------- next part --------------
From c92f2f4a227b6d8897de00bec03d4f9c7068c993 Mon Sep 17 00:00:00 2001
From: Michael Adam <obnox at samba.org>
Date: Wed, 11 Jun 2014 15:49:33 +0200
Subject: [PATCH 1/2] WIP: fix link problem with standard lib paths via
 pkg-config (e.g.)

---
 buildtools/wafadmin/Tools/ccroot.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/buildtools/wafadmin/Tools/ccroot.py b/buildtools/wafadmin/Tools/ccroot.py
index 264bdc7..3ab856b 100644
--- a/buildtools/wafadmin/Tools/ccroot.py
+++ b/buildtools/wafadmin/Tools/ccroot.py
@@ -466,11 +466,23 @@ def apply_obj_vars(self):
 	if v['FULLSTATIC']:
 		v.append_value('LINKFLAGS', v['FULLSTATIC_MARKER'])
 
+	def is_wellknown_path(p):
+		wp = ['/usr/lib', '/usr/lib/x86_64-linux-gnu']
+		for w in wp:
+			if w == p:
+				print "wellknown[%s]" % p
+				return True
+		return False
+
 	for i in v['RPATH']:
+		if is_wellknown_path(i):
+			continue
 		if i and rpath_st:
 			app('LINKFLAGS', rpath_st % i)
 
 	for i in v['LIBPATH']:
+		if is_wellknown_path(i):
+			continue
 		app('LINKFLAGS', libpath_st % i)
 		app('LINKFLAGS', staticlibpath_st % i)
 
-- 
1.9.1


From 94e69d90a53d90033c312b66cab1ab03b6de7f4c Mon Sep 17 00:00:00 2001
From: Michael Adam <obnox at samba.org>
Date: Wed, 11 Jun 2014 16:27:48 +0200
Subject: [PATCH 2/2] WIP: fix link problem with standard lib paths via
 pkg-config (e.g.)

part 2
---
 buildtools/wafadmin/Tools/config_c.py | 17 ++++++++++++++---
 buildtools/wafadmin/Tools/python.py   |  2 +-
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/buildtools/wafadmin/Tools/config_c.py b/buildtools/wafadmin/Tools/config_c.py
index d0bc617..98d43fb 100644
--- a/buildtools/wafadmin/Tools/config_c.py
+++ b/buildtools/wafadmin/Tools/config_c.py
@@ -42,6 +42,14 @@ int main() {
 def parse_flags(line, uselib, env):
 	"""pkg-config still has bugs on some platforms, and there are many -config programs, parsing flags is necessary :-/"""
 
+	def is_wellknown_path(p):
+		wp = ['/usr/lib', '/usr/lib/x86_64-linux-gnu']
+		for w in wp:
+			if w == p:
+				print "parse_flags[%s]/wellknown[%s]" % (uselib, p)
+				return True
+		return False
+
 	lst = shlex.split(line)
 	while lst:
 		x = lst.pop(0)
@@ -60,7 +68,8 @@ def parse_flags(line, uselib, env):
 			app('LIB_' + uselib, ot)
 		elif st == '-L':
 			if not ot: ot = lst.pop(0)
-			app('LIBPATH_' + uselib, ot)
+			if not is_wellknown_path(ot):
+				app('LIBPATH_' + uselib, ot)
 		elif x == '-pthread' or x.startswith('+'):
 			app('CCFLAGS_' + uselib, x)
 			app('CXXFLAGS_' + uselib, x)
@@ -83,9 +92,11 @@ def parse_flags(line, uselib, env):
 		# in too old versions of our internal libs.
 		#
 		elif x.startswith('-Wl,-R'):
-			app('RPATH_' + uselib, x[6:])
+			if not is_wellknown_path(x[6:]):
+				app('RPATH_' + uselib, x[6:])
 		elif x.startswith('-Wl,-rpath,'):
-			app('RPATH_' + uselib, x[11:])
+			if not is_wellknown_path(x[11:]):
+				app('RPATH_' + uselib, x[11:])
 		elif x.startswith('-Wl'):
 			app('LINKFLAGS_' + uselib, x)
 		elif x.startswith('-m') or x.startswith('-f'):
diff --git a/buildtools/wafadmin/Tools/python.py b/buildtools/wafadmin/Tools/python.py
index 35c61c2..17095d8 100644
--- a/buildtools/wafadmin/Tools/python.py
+++ b/buildtools/wafadmin/Tools/python.py
@@ -249,7 +249,7 @@ MACOSX_DEPLOYMENT_TARGET = %r
 		result = conf.check(lib=name, uselib='PYEMBED', libpath=path)
 
 	if result:
-		env['LIBPATH_PYEMBED'] = path
+		#env['LIBPATH_PYEMBED'] = path
 		env.append_value('LIB_PYEMBED', name)
 	else:
 		conf.log.write("\n\n### LIB NOT FOUND\n")
-- 
1.9.1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20140617/95986e60/attachment.pgp>