[PATCHES] Build pytalloc for two Python versions at once, port to py3
Petr Viktorin
pviktori at redhat.com
Wed Jan 28 09:42:04 MST 2015
Hello,
Here are buildsystem changes needed to build pytalloc for both Python 2
and Python 3 at the same time. This is enabled by configuring with
"--extra-python=/usr/bin/python3". I'm including the Python3 porting
patches I sent earlier, as there are minor changes.
How this works: The configure step is done using the existing waf
machinery, by swapping the Python-related env variables between a set
for Python2 (the "main Python") and one for Python3 ("extra python").
During build, new features for the "extra python" are used, which leads
to some duplication.
This could be used to build for e.g. python3.4 & python3.5 at once, but
I haven't tested that. If this is useful in the future I can work on it
more, otherwise all the extrapython stuff can be dropped after the
transition to Python 3.
One thing that would be required for coexistence of the two versions is
ABI-tagging any Python3 extension shared library, e.g.
"libpytalloc-util.cpython-34m.so". I think that's a good idea, but let's
hear your opinions.
The new configure option is added to all of Samba since it affects all
of the build, even though only talloc uses it. Maybe it should only be
used in individual libraries' configure scripts?
Anything that talloc doesn't need (like binaries or .py files) is not
addressed here – that'd be part of porting the bits that need it.
--
Petr Viktorin
-------------- next part --------------
From f19d8e575e9f911b3366cb5c25fca50fa898e2fe Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 12 Nov 2014 16:53:33 +0100
Subject: [PATCH 01/11] buildtools: Honor LDVERSION when looking for Python
library
Since Python 3.2, Python .so files are tagged for ABI compatibility,
so the library name is something like libpython3.4m.so (note the 'm').
This information is found in distutils.sysconfig.get_config_var('LDVERSION')
This fixes waf issue 1405 (https://code.google.com/p/waf/issues/detail?id=1405)
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafadmin/Tools/python.py | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/buildtools/wafadmin/Tools/python.py b/buildtools/wafadmin/Tools/python.py
index 35c61c2..e05f438 100644
--- a/buildtools/wafadmin/Tools/python.py
+++ b/buildtools/wafadmin/Tools/python.py
@@ -170,10 +170,10 @@ def check_python_headers(conf, mandatory=True):
try:
# Get some python configuration variables using distutils
- v = 'prefix SO SYSLIBS LDFLAGS SHLIBS LIBDIR LIBPL INCLUDEPY Py_ENABLE_SHARED MACOSX_DEPLOYMENT_TARGET'.split()
+ v = 'prefix SO SYSLIBS LDFLAGS SHLIBS LIBDIR LIBPL INCLUDEPY Py_ENABLE_SHARED MACOSX_DEPLOYMENT_TARGET LDVERSION'.split()
(python_prefix, python_SO, python_SYSLIBS, python_LDFLAGS, python_SHLIBS,
python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED,
- python_MACOSX_DEPLOYMENT_TARGET) = \
+ python_MACOSX_DEPLOYMENT_TARGET, python_LDVERSION) = \
_get_python_variables(python, ["get_config_var('%s') or ''" % x for x in v],
['from distutils.sysconfig import get_config_var'])
except RuntimeError:
@@ -190,8 +190,10 @@ python_LIBPL = %r
INCLUDEPY = %r
Py_ENABLE_SHARED = %r
MACOSX_DEPLOYMENT_TARGET = %r
+LDVERSION = %r
""" % (python, python_prefix, python_SO, python_SYSLIBS, python_LDFLAGS, python_SHLIBS,
- python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED, python_MACOSX_DEPLOYMENT_TARGET))
+ python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED, python_MACOSX_DEPLOYMENT_TARGET,
+ python_LDVERSION))
# Allow some python overrides from env vars for cross-compiling
os_env = dict(os.environ)
@@ -230,7 +232,9 @@ MACOSX_DEPLOYMENT_TARGET = %r
parse_flags(python_LDFLAGS, 'PYEMBED', env)
result = False
- name = 'python' + env['PYTHON_VERSION']
+ if not python_LDVERSION:
+ python_LDVERSION = env['PYTHON_VERSION']
+ name = 'python' + python_LDVERSION
if python_LIBDIR is not None:
path = [python_LIBDIR]
@@ -245,7 +249,7 @@ MACOSX_DEPLOYMENT_TARGET = %r
if not result:
conf.log.write("\n\n# try again with -L$prefix/libs, and pythonXY name rather than pythonX.Y (win32)\n")
path = [os.path.join(python_prefix, "libs")]
- name = 'python' + env['PYTHON_VERSION'].replace('.', '')
+ name = 'python' + python_LDVERSION.replace('.', '')
result = conf.check(lib=name, uselib='PYEMBED', libpath=path)
if result:
--
2.1.0
From 768c85137d39166b8cb6380f9715356c1fb9607f Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 12 Nov 2014 19:49:45 +0100
Subject: [PATCH 02/11] buildtools: Use all of pyext_PATTERN in
map_shlib_extension
In Python 3, C extension module filenames have an ABI tag;
the pyext_PATTERN is e.g. "%s.cpython-34m.so".
The build system was only using the last dot-separated element
of that extension (the ".so").
Use the whole extension when constructing the final filename.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index 9ac1066..8c20ff3 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -577,7 +577,7 @@ def map_shlib_extension(ctx, name, python=False):
return name
(root1, ext1) = os.path.splitext(name)
if python:
- (root2, ext2) = os.path.splitext(ctx.env.pyext_PATTERN)
+ return ctx.env.pyext_PATTERN % root1
else:
(root2, ext2) = os.path.splitext(ctx.env.shlib_PATTERN)
return root1+ext2
--
2.1.0
From dd1daec2e2e83a332990338dd9ad464545f414e2 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 16 Jan 2015 13:02:37 +0100
Subject: [PATCH 03/11] buildtools: Use 'pyfeature' instead of pyembed and
pyext arguments
This will allow defining other Python-related features, which will be
passed through the stack the same way as the existing two.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 2 +-
buildtools/wafsamba/wafsamba.py | 37 +++++++++++++++++--------------------
lib/ldb/wscript | 2 +-
lib/talloc/wscript | 2 +-
python/wscript_build | 4 ++--
source4/lib/policy/wscript_build | 2 +-
source4/librpc/wscript_build | 2 +-
source4/param/wscript_build | 4 ++--
source4/smbd/wscript_build | 2 +-
source4/torture/wscript_build | 2 +-
source4/utils/wscript_build | 2 +-
source4/web_server/wscript_build | 4 ++--
testsuite/headers/wscript_build | 2 +-
13 files changed, 32 insertions(+), 35 deletions(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 1ec2f7b..0485297 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -59,7 +59,7 @@ def SAMBA_PYTHON(bld, name,
vars=vars,
realname=realname,
link_name=link_name,
- pyext=True,
+ pyfeature='pyext',
target_type='PYTHON',
install_path='${PYTHONARCHDIR}',
allow_undefined_symbols=True,
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index c054315..1beb026 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -126,8 +126,7 @@ def SAMBA_LIBRARY(bld, libname, source,
subdir=None,
install_path=None,
install=True,
- pyembed=False,
- pyext=False,
+ pyfeature=None,
target_type='LIBRARY',
bundled_extension=False,
bundled_name=None,
@@ -193,8 +192,7 @@ def SAMBA_LIBRARY(bld, libname, source,
depends_on = depends_on,
hide_symbols = hide_symbols,
allow_warnings = allow_warnings,
- pyembed = pyembed,
- pyext = pyext,
+ pyfeature = pyfeature,
local_include = local_include,
global_include = global_include)
@@ -241,10 +239,8 @@ def SAMBA_LIBRARY(bld, libname, source,
ldflags = TO_LIST(ldflags)
features = 'cc cshlib symlink_lib install_lib'
- if pyext:
- features += ' pyext'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
if abi_directory:
features += ' abi_check'
@@ -336,7 +332,7 @@ def SAMBA_BINARY(bld, binname, source,
local_include=True,
global_include=True,
subsystem_name=None,
- pyembed=False,
+ pyfeature=None,
vars=None,
subdir=None,
install=True,
@@ -352,8 +348,8 @@ def SAMBA_BINARY(bld, binname, source,
return
features = 'cc cprogram symlink_bin install_bin'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
obj_target = binname + '.objlist'
@@ -379,6 +375,10 @@ def SAMBA_BINARY(bld, binname, source,
# first create a target for building the object files for this binary
# by separating in this way, we avoid recompiling the C files
# separately for the install binary and the build binary
+ if pyfeature == 'pyembed':
+ subsystem_pyfeature = 'pyext'
+ else:
+ subsystem_pyfeature = None
bld.SAMBA_SUBSYSTEM(obj_target,
source = source,
deps = deps,
@@ -390,7 +390,7 @@ def SAMBA_BINARY(bld, binname, source,
local_include = local_include,
global_include = global_include,
use_hostcc = use_hostcc,
- pyext = pyembed,
+ pyfeature = subsystem_pyfeature,
use_global_deps= use_global_deps)
bld.SET_BUILD_GROUP(group)
@@ -438,7 +438,7 @@ def SAMBA_MODULE(bld, modname, source,
vars=None,
subdir=None,
enabled=True,
- pyembed=False,
+ pyfeature=None,
manpages=None,
allow_undefined_symbols=False,
allow_warnings=False
@@ -508,7 +508,7 @@ def SAMBA_MODULE(bld, modname, source,
bundled_name=build_name,
link_name=build_link_name,
install_path="${MODULESDIR}/%s" % subsystem,
- pyembed=pyembed,
+ pyfeature=pyfeature,
manpages=manpages,
allow_undefined_symbols=allow_undefined_symbols,
allow_warnings=allow_warnings
@@ -544,8 +544,7 @@ def SAMBA_SUBSYSTEM(bld, modname, source,
subdir=None,
hide_symbols=False,
allow_warnings=False,
- pyext=False,
- pyembed=False):
+ pyfeature=None):
'''define a Samba subsystem'''
if not enabled:
@@ -576,10 +575,8 @@ def SAMBA_SUBSYSTEM(bld, modname, source,
bld.SET_BUILD_GROUP(group)
features = 'cc'
- if pyext:
- features += ' pyext'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
t = bld(
features = features,
diff --git a/lib/ldb/wscript b/lib/ldb/wscript
index 6391e74..fa12eb5 100755
--- a/lib/ldb/wscript
+++ b/lib/ldb/wscript
@@ -129,7 +129,7 @@ def build(bld):
vnum=VERSION,
private_library=private_library,
pc_files='pyldb-util.pc',
- pyembed=True,
+ pyfeature='pyembed',
abi_directory='ABI',
abi_match='pyldb_*')
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 986492c..221f736 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -116,7 +116,7 @@ def build(bld):
bld.SAMBA_LIBRARY('pytalloc-util',
source='pytalloc_util.c',
public_deps='talloc',
- pyembed=True,
+ pyfeature='pyembed',
vnum=VERSION,
hide_symbols=True,
abi_directory='ABI',
diff --git a/python/wscript_build b/python/wscript_build
index a40b583..56d671e 100644
--- a/python/wscript_build
+++ b/python/wscript_build
@@ -5,14 +5,14 @@ bld.SAMBA_LIBRARY('samba_python',
deps='LIBPYTHON pytalloc-util pyrpc_util',
grouping_library=True,
private_library=True,
- pyembed=True)
+ pyfeature='pyembed')
bld.SAMBA_SUBSYSTEM('LIBPYTHON',
source='modules.c',
public_deps='',
init_function_sentinel='{NULL,NULL}',
deps='talloc',
- pyext=True,
+ pyfeature='pyext',
)
diff --git a/source4/lib/policy/wscript_build b/source4/lib/policy/wscript_build
index b8ba638..8d5ea59 100644
--- a/source4/lib/policy/wscript_build
+++ b/source4/lib/policy/wscript_build
@@ -5,7 +5,7 @@ bld.SAMBA_LIBRARY('samba-policy',
pc_files='samba-policy.pc',
public_deps='ldb samba-net',
vnum='0.0.1',
- pyembed=True,
+ pyfeature='pyembed',
public_headers='policy.h'
)
diff --git a/source4/librpc/wscript_build b/source4/librpc/wscript_build
index 9b96437..d5d9b06 100755
--- a/source4/librpc/wscript_build
+++ b/source4/librpc/wscript_build
@@ -143,7 +143,7 @@ bld.SAMBA_LIBRARY('dcerpc',
bld.SAMBA_SUBSYSTEM('pyrpc_util',
source='rpc/pyrpc_util.c',
public_deps='pytalloc-util pyparam_util dcerpc MESSAGING',
- pyext=True,
+ pyfeature='pyext',
)
diff --git a/source4/param/wscript_build b/source4/param/wscript_build
index 4585a83..87f2eee 100644
--- a/source4/param/wscript_build
+++ b/source4/param/wscript_build
@@ -3,7 +3,7 @@
bld.SAMBA_SUBSYSTEM('PROVISION',
source='provision.c pyparam.c',
deps='LIBPYTHON pyparam_util ldb pytalloc-util pyldb-util',
- pyext=True,
+ pyfeature='pyext',
)
@@ -50,7 +50,7 @@ bld.SAMBA_SUBSYSTEM('param_options',
bld.SAMBA_SUBSYSTEM('pyparam_util',
source='pyparam_util.c',
deps='LIBPYTHON samba-hostconfig',
- pyext=True,
+ pyfeature='pyext',
)
bld.SAMBA_LIBRARY('shares',
diff --git a/source4/smbd/wscript_build b/source4/smbd/wscript_build
index 12d4e8b..42e442c 100644
--- a/source4/smbd/wscript_build
+++ b/source4/smbd/wscript_build
@@ -22,7 +22,7 @@ bld.SAMBA_BINARY('samba',
subsystem_name='service',
deps='''events process_model service samba-hostconfig samba-util POPT_SAMBA
popt gensec registry ntptr ntvfs share cluster COMMON_SCHANNEL SECRETS''',
- pyembed=True,
+ pyfeature='pyembed',
install_path='${SBINDIR}',
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
diff --git a/source4/torture/wscript_build b/source4/torture/wscript_build
index bbb9e9a..7f746ca 100755
--- a/source4/torture/wscript_build
+++ b/source4/torture/wscript_build
@@ -172,7 +172,7 @@ bld.SAMBA_BINARY('smbtorture',
manpages='man/smbtorture.1',
public_headers='smbtorture.h',
deps='torturemain torture popt POPT_SAMBA POPT_CREDENTIALS dcerpc LIBCLI_SMB SMBREADLINE ' + TORTURE_MODULES,
- pyembed=True
+ pyfeature='pyembed',
)
bld.SAMBA_BINARY('gentest',
diff --git a/source4/utils/wscript_build b/source4/utils/wscript_build
index 046e237..cd12da3 100644
--- a/source4/utils/wscript_build
+++ b/source4/utils/wscript_build
@@ -6,7 +6,7 @@ bld.SAMBA_BINARY('ntlm_auth4',
deps='''samba-hostconfig samba-util popt
POPT_SAMBA POPT_CREDENTIALS gensec LIBCLI_RESOLVE
auth4 NTLMSSP_COMMON MESSAGING events service''',
- pyembed=True,
+ pyfeature='pyembed',
install=False
)
diff --git a/source4/web_server/wscript_build b/source4/web_server/wscript_build
index b845067..1c44785 100644
--- a/source4/web_server/wscript_build
+++ b/source4/web_server/wscript_build
@@ -3,7 +3,7 @@
bld.SAMBA_SUBSYSTEM('WEB_WSGI',
source='wsgi.c',
- pyext=True,
+ pyfeature='pyext',
deps='talloc LIBTSOCKET',
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
@@ -14,7 +14,7 @@ bld.SAMBA_MODULE('service_web',
subsystem='service',
init_function='server_service_web_init',
deps='LIBTLS process_model LIBPYTHON WEB_WSGI',
- pyembed=True,
+ pyfeature='pyembed',
internal_module=False,
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
diff --git a/testsuite/headers/wscript_build b/testsuite/headers/wscript_build
index f612ad9..f201b05 100644
--- a/testsuite/headers/wscript_build
+++ b/testsuite/headers/wscript_build
@@ -41,7 +41,7 @@ if bld.env.DEVELOPER_MODE:
bld.SAMBA_BINARY('test_headers',
source='test_headers.c',
includes="#include/public",
- pyembed=True,
+ pyfeature='pyembed',
cflags=cflags,
local_include=True,
global_include=False,
--
2.1.0
From 001431c7f2473d6472216d3e547407af379a04eb Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 14:22:22 +0100
Subject: [PATCH 04/11] buildtools: Add --extra-python configure option
This will allow building Python support for two different Python versions
at the same time.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 4 ++++
buildtools/wafsamba/wscript | 8 ++++++++
2 files changed, 12 insertions(+)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 0485297..da5baa3 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -20,6 +20,10 @@ def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
conf.check_python_headers(mandatory)
conf.env["python_headers_checked"] = "yes"
+
+ if conf.env['EXTRAPYTHON_VERSION'] == conf.env['PYTHON_VERSION']:
+ raise Utils.WafError("extrapython %s is same as main python %s" % (
+ conf.env['EXTRAPYTHON_VERSION'], conf.env['PYTHON_VERSION']))
else:
conf.msg("python headers", "using cache")
diff --git a/buildtools/wafsamba/wscript b/buildtools/wafsamba/wscript
index c81a7b3..ebcaea7 100755
--- a/buildtools/wafsamba/wscript
+++ b/buildtools/wafsamba/wscript
@@ -192,6 +192,12 @@ def set_options(opt):
help='tag release in git at the same time',
type='string', action='store', dest='TAG_RELEASE')
+ opt.add_option('--extra-python', type=str,
+ help=("build selected libraries for the specified "
+ "additional version of Python "
+ "(example: --extra-python=/usr/bin/python3)"),
+ metavar="PYTHON", dest='EXTRA_PYTHON', default=None)
+
@wafsamba.runonce
def configure(conf):
@@ -263,6 +269,8 @@ def configure(conf):
conf.env.AUTOCONF_HOST = Options.options.AUTOCONF_HOST
conf.env.AUTOCONF_PROGRAM_PREFIX = Options.options.AUTOCONF_PROGRAM_PREFIX
+ conf.env.EXTRA_PYTHON = Options.options.EXTRA_PYTHON
+
if (conf.env.AUTOCONF_HOST and
conf.env.AUTOCONF_BUILD and
conf.env.AUTOCONF_BUILD != conf.env.AUTOCONF_HOST):
--
2.1.0
From dcf7dbdacfcad11027483dd261bebc91d35947b3 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 17:25:07 +0100
Subject: [PATCH 05/11] buildtools: Store separate configuration for extra
python
For extrapython, we store config in a separate set of env variables.
During the configure step we swap these with the normal *PYTHON* equivalents,
use waf's regular Python tool, and then switch them all back.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 88 ++++++++++++++++++++++++++++++++++++-
1 file changed, 87 insertions(+), 1 deletion(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index da5baa3..d02a683 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -1,23 +1,109 @@
-# waf build tool for building IDL files with pidl
+import contextlib
import Build
from samba_utils import *
from samba_autoconf import *
+from Logs import warn
from Configure import conf
+# For extrapython, we store config in a separate set of env variables.
+# During the configure step we swap these with the normal *PYTHON* equivalents,
+# use waf's regular Python tool, and then switch them all back.
+EXTRAPYTHON_ENV_KEYS = (
+ ('PYTHON', 'EXTRAPYTHON'),
+ ('PYTHONDIR', 'EXTRAPYTHONDIR'),
+ ('PYTHONARCHDIR', 'EXTRAPYTHONARCHDIR'),
+ ('PYTHON_VERSION', 'EXTRAPYTHON_VERSION'),
+
+ ('CCFLAGS_PYEXT', 'CCFLAGS_EXTRAPYEXT'),
+ ('CCFLAGS_PYEMBED', 'CCFLAGS_EXTRAPYEMBED'),
+
+ ('LIBPATH_PYEXT', 'LIBPATH_EXTRAPYEXT'),
+ ('LIBPATH_PYEMBED', 'LIBPATH_EXTRAPYEMBED'),
+
+ ('LIB_PYEXT', 'LIB_EXTRAPYEXT'),
+ ('LIB_PYEMBED', 'LIB_EXTRAPYEMBED'),
+
+ ('CPPPATH_PYEXT', 'CPPPATH_EXTRAPYEXT'),
+ ('CPPPATH_PYEMBED', 'CPPPATH_EXTRAPYEMBED'),
+
+ ('LINKFLAGS_PYEXT', 'LINKFLAGS_EXTRAPYEXT'),
+ ('LINKFLAGS_PYEMBED', 'LINKFLAGS_EXTRAPYEMBED'),
+
+ ('HAVE_PYTHON_H', 'EXTRAPY_HAVE_PYTHON_H'),
+
+ ('PYTHON_CONFIG', 'EXTRAPYTHON_CONFIG'),
+
+ ('pyext_PATTERN', 'extrapyext_PATTERN'),
+ ('PYCMD', 'EXTRAPYCMD'),
+ ('PYFLAGS', 'EXTRAPYFLAGS'),
+ ('PYFLAGS_OPT', 'EXTRAPYFLAGS_OPT'),
+)
+EXTRAPYTHON_DEFINES_KEYS = (
+ ('PYTHONDIR', 'EXTRAPYTHONDIR'),
+ ('PYTHONARCHDIR', 'EXTRAPYTHONARCHDIR'),
+ ('HAVE_PYTHON_H', 'EXTRAPY_HAVE_PYTHON_H'),
+)
+
+
+ at contextlib.contextmanager
+def extrapython_map(env, keys):
+ saved = {}
+ for key, extrakey in keys:
+ if key in env:
+ saved[key] = env[key]
+ if extrakey in env:
+ env[key] = env[extrakey]
+ else:
+ env.pop(key, None)
+ yield
+ for key, extrakey in keys:
+ if key in env:
+ env[extrakey] = env[key]
+ else:
+ env.pop(extrakey, None)
+ if key in saved:
+ env[key] = saved[key]
+ else:
+ env.pop(key, None)
+
+
+ at contextlib.contextmanager
+def extrapython_env(env):
+ with extrapython_map(env, EXTRAPYTHON_ENV_KEYS):
+ with extrapython_map(env['defines'], EXTRAPYTHON_DEFINES_KEYS):
+ yield
+
+
@conf
def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
# enable tool to build python extensions
+ if conf.env['EXTRA_PYTHON']:
+ with extrapython_env(conf.env):
+ conf.env['PYTHON'] = conf.env['EXTRA_PYTHON']
+ conf.find_program('python', var='PYTHON', mandatory=True)
+ conf.check_tool('python')
+ try:
+ conf.check_python_version((3, 3, 0))
+ except Exception:
+ warn('extra-python needs to be Python 3.3 or later')
+ raise
+
conf.find_program('python', var='PYTHON', mandatory=mandatory)
conf.check_tool('python')
path_python = conf.find_program('python')
conf.env.PYTHON_SPECIFIED = (conf.env.PYTHON != path_python)
conf.check_python_version(version)
+
@conf
def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
+ pre_dict = conf.env.get_merged_dict()
+ if conf.env['EXTRA_PYTHON']:
+ with extrapython_env(conf.env):
+ conf.check_python_headers(mandatory=True)
conf.check_python_headers(mandatory)
conf.env["python_headers_checked"] = "yes"
--
2.1.0
From 6d068c9819b8b0de8f2fd34ff254755b0323ae74 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 16 Jan 2015 14:52:26 +0100
Subject: [PATCH 06/11] buildtools: Add features and type for extra-python
support
This allows support for an additional version of Python to be built.
Two additional features (extrapyext & extrapyembed), and a target type
(EXTRAPYTHON) are added.
They work the same way as the "non-extra" variants, but use configuration
specific to the extra-python version.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_deps.py | 28 +++++++++++-----------
buildtools/wafsamba/samba_install.py | 8 ++++---
buildtools/wafsamba/samba_python.py | 45 ++++++++++++++++++++++++++++++++----
buildtools/wafsamba/samba_utils.py | 17 +++++++++-----
buildtools/wafsamba/stale_files.py | 2 ++
buildtools/wafsamba/symbols.py | 19 +++++++++++----
buildtools/wafsamba/wafsamba.py | 11 +++++----
7 files changed, 95 insertions(+), 35 deletions(-)
diff --git a/buildtools/wafsamba/samba_deps.py b/buildtools/wafsamba/samba_deps.py
index 3be9956..6948d65 100644
--- a/buildtools/wafsamba/samba_deps.py
+++ b/buildtools/wafsamba/samba_deps.py
@@ -72,7 +72,7 @@ def build_dependencies(self):
the full dependency list for a target until we have all of the targets declared.
'''
- if self.samba_type in ['LIBRARY', 'BINARY', 'PYTHON']:
+ if self.samba_type in ['LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON']:
self.uselib = list(self.final_syslibs)
self.uselib_local = list(self.final_libs)
self.add_objects = list(self.final_objects)
@@ -279,7 +279,7 @@ def check_duplicate_sources(bld, tgt_list):
# build a list of targets that each source file is part of
for t in tgt_list:
sources = []
- if not targets[t.sname] in [ 'LIBRARY', 'BINARY', 'PYTHON' ]:
+ if not targets[t.sname] in [ 'LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON' ]:
continue
for obj in t.add_objects:
t2 = t.bld.name_to_obj(obj, bld.env)
@@ -312,7 +312,7 @@ def check_orphaned_targets(bld, tgt_list):
if getattr(t, 'samba_used', False):
continue
type = target_dict[t.sname]
- if not type in ['BINARY', 'LIBRARY', 'MODULE', 'ET', 'PYTHON']:
+ if not type in ['BINARY', 'LIBRARY', 'MODULE', 'ET', 'PYTHON', 'EXTRAPYTHON']:
if re.search('^PIDL_', t.sname) is None:
Logs.warn("Target %s of type %s is unused by any other target" % (t.sname, type))
@@ -366,7 +366,7 @@ def show_final_deps(bld, tgt_list):
targets = LOCAL_CACHE(bld, 'TARGET_TYPE')
for t in tgt_list:
- if not targets[t.sname] in ['LIBRARY', 'BINARY', 'PYTHON', 'SUBSYSTEM']:
+ if not targets[t.sname] in ['LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON', 'SUBSYSTEM']:
continue
debug('deps: final dependencies for target %s: uselib=%s uselib_local=%s add_objects=%s',
t.sname, t.uselib, getattr(t, 'uselib_local', []), getattr(t, 'add_objects', []))
@@ -449,7 +449,9 @@ def build_direct_deps(bld, tgt_list):
sys.exit(1)
if targets[d] in [ 'EMPTY', 'DISABLED' ]:
continue
- if targets[d] == 'PYTHON' and targets[t.sname] != 'PYTHON' and t.sname.find('.objlist') == -1:
+ if (targets[d] in ('PYTHON', 'EXTRAPYTHON') and
+ targets[t.sname] not in ('PYTHON', 'EXTRAPYTHON') and
+ t.sname.find('.objlist') == -1):
# this check should be more restrictive, but for now we have pidl-generated python
# code that directly depends on other python modules
Logs.error('ERROR: Target %s has dependency on python module %s' % (t.sname, d))
@@ -475,7 +477,7 @@ def build_direct_deps(bld, tgt_list):
sys.exit(1)
if t2.samba_type in [ 'LIBRARY', 'MODULE' ]:
t.direct_libs.add(d)
- elif t2.samba_type in [ 'SUBSYSTEM', 'ASN1', 'PYTHON' ]:
+ elif t2.samba_type in [ 'SUBSYSTEM', 'ASN1', 'PYTHON', 'EXTRAPYTHON' ]:
t.direct_objects.add(d)
debug('deps: built direct dependencies')
@@ -669,7 +671,7 @@ def break_dependency_loops(bld, tgt_list):
if t.samba_type in ['SUBSYSTEM']:
loops[loop] = loops[loop].union(t.indirect_objects)
loops[loop] = loops[loop].union(t.direct_objects)
- if t.samba_type in ['LIBRARY','PYTHON']:
+ if t.samba_type in ['LIBRARY','PYTHON','EXTRAPYTHON']:
loops[loop] = loops[loop].union(t.indirect_libs)
loops[loop] = loops[loop].union(t.direct_libs)
if loop in loops[loop]:
@@ -717,7 +719,7 @@ def reduce_objects(bld, tgt_list):
changed = False
- for type in ['BINARY', 'PYTHON', 'LIBRARY']:
+ for type in ['BINARY', 'PYTHON', 'EXTRAPYTHON', 'LIBRARY']:
for t in tgt_list:
if t.samba_type != type: continue
# if we will indirectly link to a target then we don't need it
@@ -817,7 +819,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# find any library loops
for t in tgt_list:
- if t.samba_type in ['LIBRARY', 'PYTHON']:
+ if t.samba_type in ['LIBRARY', 'PYTHON', 'EXTRAPYTHON']:
for l in t.final_libs.copy():
t2 = bld.name_to_obj(l, bld.env)
if t.sname in t2.final_libs:
@@ -841,7 +843,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# we now need to make corrections for any library loops we broke up
# any target that depended on the target of the loop and doesn't
# depend on the source of the loop needs to get the loop source added
- for type in ['BINARY','PYTHON','LIBRARY','BINARY']:
+ for type in ['BINARY','PYTHON','EXTRAPYTHON','LIBRARY','BINARY']:
for t in tgt_list:
if t.samba_type != type: continue
for loop in loops:
@@ -874,7 +876,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# add in any syslib dependencies
for t in tgt_list:
- if not t.samba_type in ['BINARY','PYTHON','LIBRARY','SUBSYSTEM']:
+ if not t.samba_type in ['BINARY','PYTHON','EXTRAPYTHON','LIBRARY','SUBSYSTEM']:
continue
syslibs = set()
for d in t.final_objects:
@@ -891,7 +893,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# find any unresolved library loops
lib_loop_error = False
for t in tgt_list:
- if t.samba_type in ['LIBRARY', 'PYTHON']:
+ if t.samba_type in ['LIBRARY', 'PYTHON', 'EXTRAPYTHON']:
for l in t.final_libs.copy():
t2 = bld.name_to_obj(l, bld.env)
if t.sname in t2.final_libs:
@@ -935,7 +937,7 @@ def show_object_duplicates(bld, tgt_list):
Logs.info("showing duplicate objects")
for t in tgt_list:
- if not targets[t.sname] in [ 'LIBRARY', 'PYTHON' ]:
+ if not targets[t.sname] in [ 'LIBRARY', 'PYTHON', 'EXTRAPYTHON' ]:
continue
for n in getattr(t, 'final_objects', set()):
t2 = bld.name_to_obj(n, bld.env)
diff --git a/buildtools/wafsamba/samba_install.py b/buildtools/wafsamba/samba_install.py
index aa7f143..4a5780b 100644
--- a/buildtools/wafsamba/samba_install.py
+++ b/buildtools/wafsamba/samba_install.py
@@ -105,10 +105,12 @@ def install_library(self):
install_link = None
if getattr(self, 'soname', ''):
install_link = self.soname
- if getattr(self, 'samba_type', None) == 'PYTHON':
- inst_name = bld.make_libname(t.target, nolibprefix=True, python=True)
+ samba_type = getattr(self, 'samba_type', None)
+ if samba_type in ('PYTHON', 'EXTRAPYTHON'):
+ inst_name = bld.make_libname(t.target, nolibprefix=True,
+ target_type=samba_type)
else:
- inst_name = bld.make_libname(t.target)
+ inst_name = bld.make_libname(t.target)
elif self.vnum:
vnum_base = self.vnum.split('.')[0]
install_name = bld.make_libname(target_name, version=self.vnum)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index d02a683..9de169c 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -4,9 +4,37 @@ import Build
from samba_utils import *
from samba_autoconf import *
from Logs import warn
+from TaskGen import extension, before, after, feature
from Configure import conf
+
+ at feature('extrapyext')
+ at before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars', 'apply_bundle')
+ at after('vars_target_cshlib')
+def init_extrapyext(self):
+ self.default_install_path = '${PYTHONARCHDIR}'
+ self.uselib = self.to_list(getattr(self, 'uselib', ''))
+ if not 'EXTRAPYEXT' in self.uselib:
+ self.uselib.append('EXTRAPYEXT')
+
+
+ at before('apply_link', 'apply_lib_vars', 'apply_type_vars')
+ at after('apply_bundle')
+ at feature('extrapyext')
+def extrapyext_shlib_ext(self):
+ # override shlib_PATTERN set by the osx module
+ self.env['shlib_PATTERN'] = self.env['extrapyext_PATTERN']
+
+
+ at before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars')
+ at feature('extrapyembed')
+def init_extrapyembed(self):
+ self.uselib = self.to_list(getattr(self, 'uselib', ''))
+ if not 'EXTRAPYEMBED' in self.uselib:
+ self.uselib.append('EXTRAPYEMBED')
+
+
# For extrapython, we store config in a separate set of env variables.
# During the configure step we swap these with the normal *PYTHON* equivalents,
# use waf's regular Python tool, and then switch them all back.
@@ -124,7 +152,8 @@ def SAMBA_PYTHON(bld, name,
init_function_sentinel=None,
local_include=True,
vars=None,
- enabled=True):
+ enabled=True,
+ extra_python=False):
'''build a python extension for Samba'''
# when we support static python modules we'll need to gather
@@ -139,6 +168,14 @@ def SAMBA_PYTHON(bld, name,
else:
link_name = None
+ if extra_python:
+ pyfeature = 'extrapyext'
+ target_type = 'EXTRAPYTHON'
+ install_path = '${EXTRAPYTHONARCHDIR}'
+ else:
+ pyfeature = 'pyext'
+ target_type = 'PYTHON'
+ install_path = '${PYTHONARCHDIR}'
bld.SAMBA_LIBRARY(name,
source=source,
deps=deps,
@@ -149,9 +186,9 @@ def SAMBA_PYTHON(bld, name,
vars=vars,
realname=realname,
link_name=link_name,
- pyfeature='pyext',
- target_type='PYTHON',
- install_path='${PYTHONARCHDIR}',
+ pyfeature=pyfeature,
+ target_type=target_type,
+ install_path=install_path,
allow_undefined_symbols=True,
allow_warnings=True,
enabled=enabled)
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index 8c20ff3..c8cfa99 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -568,7 +568,7 @@ def reconfigure(ctx):
Scripting.check_configured(bld)
-def map_shlib_extension(ctx, name, python=False):
+def map_shlib_extension(ctx, name, target_type=None):
'''map a filename with a shared library extension of .so to the real shlib name'''
if name is None:
return None
@@ -576,8 +576,10 @@ def map_shlib_extension(ctx, name, python=False):
# some libraries have specified versions in the wscript rule
return name
(root1, ext1) = os.path.splitext(name)
- if python:
+ if target_type == 'PYTHON':
return ctx.env.pyext_PATTERN % root1
+ elif target_type == 'EXTRAPYTHON':
+ return ctx.env.extrapyext_PATTERN % root1
else:
(root2, ext2) = os.path.splitext(ctx.env.shlib_PATTERN)
return root1+ext2
@@ -591,15 +593,18 @@ def apply_pattern(filename, pattern):
basename = os.path.basename(filename)
return os.path.join(dirname, pattern % basename)
-def make_libname(ctx, name, nolibprefix=False, version=None, python=False):
+def make_libname(ctx, name, nolibprefix=False, version=None, target_type=None):
"""make a library filename
Options:
nolibprefix: don't include the lib prefix
version : add a version number
- python : if we should use python module name conventions"""
+ target_type: 'PYTHON' or 'EXTRAPYTHON' if we should use python module name conventions
+ """
- if python:
+ if target_type == 'PYTHON':
libname = apply_pattern(name, ctx.env.pyext_PATTERN)
+ elif target_type == 'EXTRAPYTHON':
+ libname = apply_pattern(name, ctx.env.extrapyext_PATTERN)
else:
libname = apply_pattern(name, ctx.env.shlib_PATTERN)
if nolibprefix and libname[0:3] == 'lib':
@@ -626,7 +631,7 @@ def get_tgt_list(bld):
tgt_list = []
for tgt in targets:
type = targets[tgt]
- if not type in ['SUBSYSTEM', 'MODULE', 'BINARY', 'LIBRARY', 'ASN1', 'PYTHON']:
+ if not type in ['SUBSYSTEM', 'MODULE', 'BINARY', 'LIBRARY', 'ASN1', 'PYTHON', 'EXTRAPYTHON']:
continue
t = bld.name_to_obj(tgt, bld.env)
if t is None:
diff --git a/buildtools/wafsamba/stale_files.py b/buildtools/wafsamba/stale_files.py
index 2dd08e1..8c5d5ee 100644
--- a/buildtools/wafsamba/stale_files.py
+++ b/buildtools/wafsamba/stale_files.py
@@ -71,6 +71,8 @@ def replace_refill_task_list(self):
t = samba_utils.apply_pattern(t, bld.env.shlib_PATTERN)
if ttype == 'PYTHON':
t = samba_utils.apply_pattern(t, bld.env.pyext_PATTERN)
+ elif ttype == 'EXTRAPYTHON':
+ t = samba_utils.apply_pattern(t, bld.env.extrapyext_PATTERN)
p = os.path.join(x.path.abspath(bld.env), t)
p = os.path.normpath(p)
expected.append(p)
diff --git a/buildtools/wafsamba/symbols.py b/buildtools/wafsamba/symbols.py
index daa18b9..6750a8a 100644
--- a/buildtools/wafsamba/symbols.py
+++ b/buildtools/wafsamba/symbols.py
@@ -278,7 +278,7 @@ def build_library_dict(bld, tgt_list):
bld.env.library_dict = {}
for t in tgt_list:
- if t.samba_type in [ 'LIBRARY', 'PYTHON' ]:
+ if t.samba_type in [ 'LIBRARY', 'PYTHON', 'EXTRAPYTHON' ]:
linkpath = os.path.realpath(t.link_task.outputs[0].abspath(bld.env))
bld.env.library_dict[linkpath] = t.sname
@@ -293,10 +293,12 @@ def build_syslib_sets(bld, tgt_list):
syslibs = {}
objmap = {}
for t in tgt_list:
- if getattr(t, 'uselib', []) and t.samba_type in [ 'LIBRARY', 'BINARY', 'PYTHON' ]:
+ if getattr(t, 'uselib', []) and t.samba_type in [ 'LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON' ]:
for lib in t.uselib:
if lib in ['PYEMBED', 'PYEXT']:
lib = "python"
+ elif lib in ['EXTRAPYEMBED', 'EXTRAPYEXT']:
+ lib = "python" + bld.env.EXTRAPYTHON_VERSION
if not lib in syslibs:
syslibs[lib] = []
syslibs[lib].append(t)
@@ -356,7 +358,8 @@ def build_autodeps(bld, t):
if t.in_library == depname:
# no need to depend on the library we are part of
continue
- if depname[0] in ['c', 'python']:
+ if depname[0] in ['c', 'python',
+ 'python' + bld.env.EXTRAPYTHON_VERSION]:
# these don't go into autodeps
continue
if targets[depname[0]] in [ 'SYSLIB' ]:
@@ -484,8 +487,14 @@ def check_syslib_dependencies(bld, t):
features = TO_LIST(t.features)
if 'pyembed' in features or 'pyext' in features:
- if 'python' in bld.env.public_symbols:
- t.unsatisfied_symbols = t.unsatisfied_symbols.difference(bld.env.public_symbols['python'])
+ python_libname = 'python'
+ elif 'extrapyembed' in features or 'extrapyext' in features:
+ python_libname = 'python' + bld.env.EXTRAPYTHON_VERSION
+ else:
+ python_libname = None
+ if python_libname and python_libname in bld.env.public_symbols:
+ py_symbols = bld.env.public_symbols[python_libname]
+ t.unsatisfied_symbols = t.unsatisfied_symbols.difference(py_symbols)
needed = {}
for sym in t.unsatisfied_symbols:
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index 1beb026..29c4f88 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -207,11 +207,12 @@ def SAMBA_LIBRARY(bld, libname, source,
deps = TO_LIST(deps)
deps.append(obj_target)
- realname = bld.map_shlib_extension(realname, python=(target_type=='PYTHON'))
- link_name = bld.map_shlib_extension(link_name, python=(target_type=='PYTHON'))
+ realname = bld.map_shlib_extension(realname, target_type=target_type)
+ link_name = bld.map_shlib_extension(link_name, target_type=target_type)
# we don't want any public libraries without version numbers
- if (not private_library and target_type != 'PYTHON' and not realname):
+ if (not private_library and not realname and
+ target_type not in ('PYTHON', 'EXTRAPYTHON')):
if vnum is None and soname is None:
raise Utils.WafError("public library '%s' must have a vnum" %
libname)
@@ -224,7 +225,7 @@ def SAMBA_LIBRARY(bld, libname, source,
if bundled_name is not None:
pass
- elif target_type == 'PYTHON' or realname or not private_library:
+ elif target_type in ('PYTHON', 'EXTRAPYTHON') or realname or not private_library:
if keep_underscore:
bundled_name = libname
else:
@@ -377,6 +378,8 @@ def SAMBA_BINARY(bld, binname, source,
# separately for the install binary and the build binary
if pyfeature == 'pyembed':
subsystem_pyfeature = 'pyext'
+ elif pyfeature == 'extrapyembed':
+ subsystem_pyfeature = 'extrapyext'
else:
subsystem_pyfeature = None
bld.SAMBA_SUBSYSTEM(obj_target,
--
2.1.0
From 71708652fcc8440ebc4d6fc892a517a766e403c1 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 23 Jan 2015 16:32:21 +0100
Subject: [PATCH 07/11] buildtools: Don't require pkgconfig file or headers for
extrapython
For system configuration one should use the system version of Python,
not extrapython.
Public headers should be the same for all versions, so the ones for
the main Python version should be used.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/wafsamba.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index 29c4f88..cd301b5 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -216,10 +216,10 @@ def SAMBA_LIBRARY(bld, libname, source,
if vnum is None and soname is None:
raise Utils.WafError("public library '%s' must have a vnum" %
libname)
- if pc_files is None:
+ if pc_files is None and pyfeature != 'extrapyembed':
raise Utils.WafError("public library '%s' must have pkg-config file" %
libname)
- if public_headers is None:
+ if public_headers is None and pyfeature != 'extrapyembed':
raise Utils.WafError("public library '%s' must have header files" %
libname)
--
2.1.0
From a517242cf6e841eb10c80fdf67227444813aa919 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 23 Jan 2015 17:02:55 +0100
Subject: [PATCH 08/11] buildtools: Enable adding ABI flags to Python utilities
This allows shared libraries to be named, for example,
"libpytalloc-util.cpython-34m.so". The ABI flag enables
libraries for several Python versions to co-exist on
a single system.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 9de169c..5d589e9 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -1,3 +1,4 @@
+import os
import contextlib
import Build
@@ -67,6 +68,8 @@ EXTRAPYTHON_ENV_KEYS = (
('PYCMD', 'EXTRAPYCMD'),
('PYFLAGS', 'EXTRAPYFLAGS'),
('PYFLAGS_OPT', 'EXTRAPYFLAGS_OPT'),
+
+ ('PYTHON_SO_ABI_FLAG', 'EXTRAPYTHON_SO_ABI_FLAG'),
)
EXTRAPYTHON_DEFINES_KEYS = (
('PYTHONDIR', 'EXTRAPYTHONDIR'),
@@ -125,14 +128,24 @@ def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
conf.check_python_version(version)
+def check_python_headers(conf, mandatory):
+ conf.check_python_headers(mandatory=mandatory)
+
+ if conf.env['PYTHON_VERSION'] > '3':
+ abi_pattern = os.path.splitext(conf.env['pyext_PATTERN'])[0]
+ conf.env['PYTHON_SO_ABI_FLAG'] = abi_pattern % ''
+ else:
+ conf.env['PYTHON_SO_ABI_FLAG'] = ''
+
+
@conf
def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
pre_dict = conf.env.get_merged_dict()
if conf.env['EXTRA_PYTHON']:
with extrapython_env(conf.env):
- conf.check_python_headers(mandatory=True)
- conf.check_python_headers(mandatory)
+ check_python_headers(conf, mandatory=True)
+ check_python_headers(conf, mandatory=mandatory)
conf.env["python_headers_checked"] = "yes"
if conf.env['EXTRAPYTHON_VERSION'] == conf.env['PYTHON_VERSION']:
@@ -194,3 +207,14 @@ def SAMBA_PYTHON(bld, name,
enabled=enabled)
Build.BuildContext.SAMBA_PYTHON = SAMBA_PYTHON
+
+
+def pyembed_libname(bld, name, extrapython=False):
+ if extrapython:
+ return name + bld.env['EXTRAPYTHON_SO_ABI_FLAG']
+ else:
+ return name + bld.env['PYTHON_SO_ABI_FLAG']
+
+ return name
+
+Build.BuildContext.pyembed_libname = pyembed_libname
--
2.1.0
From aa499fc959107587de4197a649963b8bdfdfb4cd Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 14:07:09 +0100
Subject: [PATCH 09/11] talloc build: Use SAMBA_CHECK_PYTHON for finding Python
Previously the code repeated most of SAMBA_CHECK_PYTHON explicitly.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/wscript | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 221f736..6e62a9e 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -59,9 +59,7 @@ def configure(conf):
if not conf.env.disable_python:
# also disable if we don't have the python libs installed
- conf.find_program('python', var='PYTHON')
- conf.check_tool('python')
- conf.check_python_version((2,4,2))
+ conf.SAMBA_CHECK_PYTHON(mandatory=False, version=(2,4,2))
conf.SAMBA_CHECK_PYTHON_HEADERS(mandatory=False)
if not conf.env.HAVE_PYTHON_H:
Logs.warn('Disabling pytalloc-util as python devel libs not found')
--
2.1.0
From 1e7eff6256809c7a2de3efde045ee53ea348e505 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 12 Nov 2014 17:39:24 +0100
Subject: [PATCH 10/11] pytalloc: Port to Python 3
- Use PyUnicode instead of PyString for repr
- Use rich comparison
Removes the deprecated tp_compare in favor of tp_richcompare.
Disparate types cannot be compared (except for == and !=),
and True or False objects are returned explicitly.
- Use Py_TYPE instead of ob_type
This changed to conform to C aliasing rules,
see http://legacy.python.org/dev/peps/pep-3123/
- Don't provide CObject creation function
A PyCapsule based replacement would be possible,
but might not be necessary considering the function is
not used much.
- Use new-style module initialization
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/pytalloc.c | 104 +++++++++++++++++++++++++++++++++++++-----
lib/talloc/pytalloc.h | 2 +
lib/talloc/pytalloc_guide.txt | 2 +
lib/talloc/pytalloc_util.c | 2 +
4 files changed, 98 insertions(+), 12 deletions(-)
diff --git a/lib/talloc/pytalloc.c b/lib/talloc/pytalloc.c
index 80196c6..9d18097 100644
--- a/lib/talloc/pytalloc.c
+++ b/lib/talloc/pytalloc.c
@@ -21,8 +21,6 @@
#include <talloc.h>
#include <pytalloc.h>
-void inittalloc(void);
-
/* print a talloc tree report for a talloc python object */
static PyObject *pytalloc_report_full(PyObject *self, PyObject *args)
{
@@ -79,8 +77,13 @@ static PyObject *pytalloc_default_repr(PyObject *obj)
pytalloc_Object *talloc_obj = (pytalloc_Object *)obj;
PyTypeObject *type = (PyTypeObject*)PyObject_Type(obj);
+#if PY_MAJOR_VERSION >= 3
+ return PyUnicode_FromFormat("<%s talloc object at 0x%p>",
+ type->tp_name, talloc_obj->ptr);
+#else
return PyString_FromFormat("<%s talloc object at 0x%p>",
type->tp_name, talloc_obj->ptr);
+#endif
}
/**
@@ -91,21 +94,56 @@ static void pytalloc_dealloc(PyObject* self)
pytalloc_Object *obj = (pytalloc_Object *)self;
assert(talloc_unlink(NULL, obj->talloc_ctx) != -1);
obj->talloc_ctx = NULL;
- self->ob_type->tp_free(self);
+ Py_TYPE(self)->tp_free(self);
}
/**
* Default (but only slightly more useful than the default) implementation of cmp.
*/
+#if PY_MAJOR_VERSION >= 3
+static PyObject* pytalloc_default_py3richcmp(PyObject *_obj1, PyObject *_obj2, int op)
+{
+ int cmp, result;
+
+ pytalloc_Object *obj1 = (pytalloc_Object *)_obj1,
+ *obj2 = (pytalloc_Object *)_obj2;
+ if (Py_TYPE(obj1) != Py_TYPE(obj2)) {
+ if (op == Py_EQ) {
+ result = obj1 == obj2;
+ } else if (op == Py_NE) {
+ result = obj1 != obj2;
+ } else {
+ Py_RETURN_NOTIMPLEMENTED;
+ }
+ } else {
+ cmp = ((char *)pytalloc_get_ptr(obj1) - (char *)pytalloc_get_ptr(obj2));
+ switch (op) {
+ case Py_LT: result = cmp < 0;
+ case Py_LE: result = cmp <= 0;
+ case Py_EQ: result = cmp == 0;
+ case Py_NE: result = cmp != 0;
+ case Py_GT: result = cmp > 0;
+ case Py_GE: result = cmp >= 0;
+ }
+ }
+ if (result) {
+ Py_RETURN_TRUE;
+ } else {
+ Py_RETURN_FALSE;
+ }
+}
+#else
static int pytalloc_default_cmp(PyObject *_obj1, PyObject *_obj2)
{
pytalloc_Object *obj1 = (pytalloc_Object *)_obj1,
*obj2 = (pytalloc_Object *)_obj2;
- if (obj1->ob_type != obj2->ob_type)
- return (obj1->ob_type - obj2->ob_type);
+ if (Py_TYPE(obj1) != Py_TYPE(obj2))
+ return (Py_TYPE(obj1) - Py_TYPE(obj2));
return ((char *)pytalloc_get_ptr(obj1) - (char *)pytalloc_get_ptr(obj2));
}
+#endif
+
static PyTypeObject TallocObject_Type = {
.tp_name = "talloc.Object",
@@ -114,21 +152,63 @@ static PyTypeObject TallocObject_Type = {
.tp_dealloc = (destructor)pytalloc_dealloc,
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
.tp_repr = pytalloc_default_repr,
+#if PY_MAJOR_VERSION >= 3
+ .tp_richcompare = pytalloc_default_py3richcmp,
+#else
.tp_compare = pytalloc_default_cmp,
+#endif
};
-void inittalloc(void)
+#define MODULE_DOC "Python wrapping of talloc-maintained objects."
+
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef moduledef = {
+ PyModuleDef_HEAD_INIT,
+ .m_name = "talloc",
+ .m_doc = MODULE_DOC,
+ .m_size = -1,
+ .m_methods = talloc_methods,
+ .m_reload = NULL,
+ .m_traverse = NULL,
+ .m_clear = NULL,
+ .m_free = NULL,
+};
+#endif
+
+
+static PyObject* module_init(void)
{
PyObject *m;
- if (PyType_Ready(&TallocObject_Type) < 0)
- return;
+ if (PyType_Ready(&TallocObject_Type) < 0) {
+ return NULL;
+ }
- m = Py_InitModule3("talloc", talloc_methods,
- "Python wrapping of talloc-maintained objects.");
- if (m == NULL)
- return;
+#if PY_MAJOR_VERSION >= 3
+ m = PyModule_Create(&moduledef);
+#else
+ m = Py_InitModule3("talloc", talloc_methods, MODULE_DOC);
+#endif
+
+ if (m == NULL) {
+ return NULL;
+ }
Py_INCREF(&TallocObject_Type);
PyModule_AddObject(m, "Object", (PyObject *)&TallocObject_Type);
+
+ return m;
+}
+
+
+#if PY_MAJOR_VERSION >= 3
+PyMODINIT_FUNC PyInit_talloc(void)
+{
+ return module_init();
+}
+#else
+void inittalloc(void)
+{
+ module_init();
}
+#endif
diff --git a/lib/talloc/pytalloc.h b/lib/talloc/pytalloc.h
index 5c3876e..608328e 100644
--- a/lib/talloc/pytalloc.h
+++ b/lib/talloc/pytalloc.h
@@ -52,6 +52,8 @@ PyObject *pytalloc_reference_ex(PyTypeObject *py_type, TALLOC_CTX *mem_ctx, void
#define pytalloc_new(type, typeobj) pytalloc_steal(typeobj, talloc_zero(NULL, type))
+#if PY_MAJOR_VERSION < 3
PyObject *pytalloc_CObject_FromTallocPtr(void *);
+#endif
#endif /* _PYTALLOC_H_ */
diff --git a/lib/talloc/pytalloc_guide.txt b/lib/talloc/pytalloc_guide.txt
index 755a52b..8abe1d4 100644
--- a/lib/talloc/pytalloc_guide.txt
+++ b/lib/talloc/pytalloc_guide.txt
@@ -126,6 +126,8 @@ use a generic VoidPtr Python type, which just provides an opaque object in
Python. The caller is responsible for incrementing the talloc reference count before calling
this function - it will dereference the talloc pointer when it is garbage collected.
+This function is only available on Python 2.
+
Debug function for talloc in Python
-----------------------------------
diff --git a/lib/talloc/pytalloc_util.c b/lib/talloc/pytalloc_util.c
index 89a093b..94ba8d2 100644
--- a/lib/talloc/pytalloc_util.c
+++ b/lib/talloc/pytalloc_util.c
@@ -102,6 +102,7 @@ static void py_cobject_talloc_free(void *ptr)
talloc_free(ptr);
}
+#if PY_MAJOR_VERSION < 3
_PUBLIC_ PyObject *pytalloc_CObject_FromTallocPtr(void *ptr)
{
if (ptr == NULL) {
@@ -109,6 +110,7 @@ _PUBLIC_ PyObject *pytalloc_CObject_FromTallocPtr(void *ptr)
}
return PyCObject_FromVoidPtr(ptr, py_cobject_talloc_free);
}
+#endif
_PUBLIC_ int pytalloc_Check(PyObject *obj)
{
--
2.1.0
From 999a219d41c6ca0c2bc29e1f251350a536e0da76 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 19 Nov 2014 14:50:07 +0100
Subject: [PATCH 11/11] pytalloc: Build for Python 3/extrapython
This enables building and installing two versions for pytalloc,
for two versions of Python, at the same time.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/pytalloc-util.pc.in | 2 +-
lib/talloc/pytalloc_guide.txt | 8 ++++++++
lib/talloc/wscript | 32 ++++++++++++++++++++++++++------
3 files changed, 35 insertions(+), 7 deletions(-)
diff --git a/lib/talloc/pytalloc-util.pc.in b/lib/talloc/pytalloc-util.pc.in
index b7426bb..b87c94e 100644
--- a/lib/talloc/pytalloc-util.pc.in
+++ b/lib/talloc/pytalloc-util.pc.in
@@ -6,6 +6,6 @@ includedir=@includedir@
Name: pytalloc-util
Description: Utility functions for using talloc objects with Python
Version: @TALLOC_VERSION@
-Libs: @LIB_RPATH@ -L${libdir} -lpytalloc-util
+Libs: @LIB_RPATH@ -L${libdir} -lpytalloc-util at PYTHON_SO_ABI_FLAG@
Cflags: -I${includedir}
URL: http://talloc.samba.org/
diff --git a/lib/talloc/pytalloc_guide.txt b/lib/talloc/pytalloc_guide.txt
index 8abe1d4..36ae5ff 100644
--- a/lib/talloc/pytalloc_guide.txt
+++ b/lib/talloc/pytalloc_guide.txt
@@ -20,6 +20,14 @@ for objects that wrap talloc-maintained memory in C. It won't write your
bindings for you but it will make it easier to write C bindings that involve
talloc, and take away some of the boiler plate.
+Python 3
+--------
+
+pytalloc can be used with Python 3. Usage from Python extension remains
+the same, but for the C utilities, the library to link to is tagged with
+Python's PEP3149 ABI tag, for example "pytalloc.cpython34m".
+To make a build for Python 3, configure with PYTHON=/usr/bin/python3.
+.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
pytalloc_Object
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 6e62a9e..2aef72b 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -111,24 +111,44 @@ def build(bld):
manpages='man/talloc.3')
if not bld.CONFIG_SET('USING_SYSTEM_PYTALLOC_UTIL') and not bld.env.disable_python:
- bld.SAMBA_LIBRARY('pytalloc-util',
+ name = bld.pyembed_libname('pytalloc-util')
+ options = dict(
source='pytalloc_util.c',
public_deps='talloc',
- pyfeature='pyembed',
vnum=VERSION,
hide_symbols=True,
abi_directory='ABI',
- abi_match='pytalloc_*',
private_library=private_library,
- public_headers='pytalloc.h',
- pc_files='pytalloc-util.pc'
+ abi_match='pytalloc_*',
)
+
+ name = bld.pyembed_libname('pytalloc-util')
+ bld.SAMBA_LIBRARY(name,
+ pyfeature='pyembed',
+ pc_files='pytalloc-util.pc',
+ public_headers='pytalloc.h',
+ **options)
bld.SAMBA_PYTHON('pytalloc',
'pytalloc.c',
- deps='talloc pytalloc-util',
+ deps='talloc ' + name,
enabled=True,
realname='talloc.so')
+ if bld.env['EXTRA_PYTHON']:
+ name = bld.pyembed_libname('pytalloc-util', extrapython=True)
+ bld.SAMBA_LIBRARY(name,
+ pyfeature='extrapyembed',
+ pc_files=None,
+ **options)
+
+ bld.SAMBA_PYTHON('extra-pytalloc',
+ 'pytalloc.c',
+ deps='talloc ' + name,
+ enabled=True,
+ realname='talloc.so',
+ extra_python=True)
+
+
def test(ctx):
'''run talloc testsuite'''
import Utils, samba_utils
--
2.1.0
More information about the samba-technical
mailing list