[PATCHES] Build pytalloc for two Python versions at once, port to py3
Petr Viktorin
pviktori at redhat.com
Thu Mar 12 05:08:48 MDT 2015
On 03/05/2015 10:08 PM, Andrew Bartlett wrote:
> On Wed, 2015-03-04 at 15:53 +0100, Jelmer Vernooij wrote:
>> On Wed, Mar 04, 2015 at 11:18:08AM +1300, Andrew Bartlett wrote:
>>> On Tue, 2015-03-03 at 22:09 +0100, Jelmer Vernooij wrote:
>>>> On Mon, Mar 02, 2015 at 12:09:25PM +0100, Petr Viktorin wrote:
>
>>>> * It shouldn't clutter the code with lots of compatibility wrappers
>>>> that make the code unreadable.
>>>
>>> I'm sorry to hear earlier in the thread that there are no efforts to
>>> have a compatibility API for the C interface, because it seems that in
>>> Samba, we already do have a good separation between bytes and unicode,
>>> because we care to be null-termination safe.
>>>
>>> That is, length-limited strings are byte arrays, null terminated strings
>>> are utf8 strings. Within those rules, and within Jelmer's stipulation
>>> above, I would like to see a library that implements the Python3 C API
>>> in terms of python2. Then we could port Samba to that, one module at a
>>> time. It wouldn't make the code unreadable, because it would be the
>>> Python3 API. I might permit a small number of #ifdefs, but these too
>>> make the code unreadable.
>> FWIW I am very skeptical that such a (python3 on top of python2 API) wrapper
>> would be possible, especially in a way that makes it less painful than #ifdefs.
>>
>> It might have been possible if the Python3 API had been designed with such
>> a compatibility layer in mind, but it hasn't.
>
> Indeed, I may be requesting a monster.
>
> Petr,
>
> The reason I'm being so awkward is that I don't like any of the options.
> - waiting forever means we:
> - push aside your valuable contributions
> - may never make the move
I don't think the second point is true; you'd be forced to make the move
eventually, most likely when Python 2 support ends in 5 years. It's good
not to rush things, but waiting forever is hardly an option -- it would
just make us rush things later.
> - waiting for things to improve:
> - you suggest is foolish, there is no indication that the python community will
> - and we don't know that someone else will be keen to do the work in the future
Also, there's a non-trivial chance that my work on Samba could drive
these improvements.
It's been suggested a CPython list that I write a porting guide for C
code. I'm quite tempted to share what I write for Samba.
> - moving now:
> - risks years of migration wrappers/ifdef etc
I will strive to make this as palatable as possible.
> - risks restricting developers to the cumbersome common subset - yet another language, essentially
That mostly applies to Python source. In the C extensions, alternatives
to the removed things have usually been ported to Python 2.6, so not
even an #ifdef is necesary. The main problem is deciding between bytes
and unicode.
> - imposes a burden on the already very small part of the Samba team with an interest in maintaining the python code
That is, unfortunately, true. On the other hand, the porting process
will find and fix some bugs as well – see commits 6b89848...c05b0a3 for
example.
> I do really appreciate your efforts, and in particular your efforts to
> add tests. That will help improve confidence. If we can do that, and
> follow the incremental development principals we use elsewhere, then I
> think we have a small hope of success.
>
> I do realise we will at some point need to do a large pass over the
> code, and I honestly don't look forward to it.
Here are updated patches for the buildsystem and talloc. This time, I
added a compatibility layer in the form of a header file.
The patches depend on pytalloc tests I sent in another thread.
As always, the buildsystem changes are big but necessary to move forward
-- I don't think an incremental process is possible without them.
If and when these patches are accepted, I'd like to take the py3compat
header file out of talloc, and release it as a separate library (as you
suggested). I'm not yet interested in figuring out the logistics of
that, but I have started writing a detailed README:
https://fedorapeople.org/~pviktori/py3compat/readme.html
I've started looking at tdb and ldb; you can see the WIP patches here:
https://fedorapeople.org/cgit/pviktori/public_git/samba.git/commit/?h=py3compat-34ad28d&id=d479cbfe08b143f65c674a005f2acc9257a30a47
https://fedorapeople.org/cgit/pviktori/public_git/samba.git/commit/?h=py3compat-34ad28d&id=34ad28d225771541e3a294bf3782be0839575148
I'm not yet interested in a review of these patches, just note that they
don't really add a mess of #defines.
--
Petr Viktorin
-------------- next part --------------
From 7482366c38583eb30e801843a704982c6ac7c53f Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 12 Nov 2014 16:53:33 +0100
Subject: [PATCH 01/13] buildtools: Honor LDVERSION when looking for Python
library
Since Python 3.2, Python .so files are tagged for ABI compatibility,
so the library name is something like libpython3.4m.so (note the 'm').
This information is found in distutils.sysconfig.get_config_var('LDVERSION')
This fixes waf issue 1405 (https://code.google.com/p/waf/issues/detail?id=1405)
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafadmin/Tools/python.py | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/buildtools/wafadmin/Tools/python.py b/buildtools/wafadmin/Tools/python.py
index 35c61c2..e05f438 100644
--- a/buildtools/wafadmin/Tools/python.py
+++ b/buildtools/wafadmin/Tools/python.py
@@ -170,10 +170,10 @@ def check_python_headers(conf, mandatory=True):
try:
# Get some python configuration variables using distutils
- v = 'prefix SO SYSLIBS LDFLAGS SHLIBS LIBDIR LIBPL INCLUDEPY Py_ENABLE_SHARED MACOSX_DEPLOYMENT_TARGET'.split()
+ v = 'prefix SO SYSLIBS LDFLAGS SHLIBS LIBDIR LIBPL INCLUDEPY Py_ENABLE_SHARED MACOSX_DEPLOYMENT_TARGET LDVERSION'.split()
(python_prefix, python_SO, python_SYSLIBS, python_LDFLAGS, python_SHLIBS,
python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED,
- python_MACOSX_DEPLOYMENT_TARGET) = \
+ python_MACOSX_DEPLOYMENT_TARGET, python_LDVERSION) = \
_get_python_variables(python, ["get_config_var('%s') or ''" % x for x in v],
['from distutils.sysconfig import get_config_var'])
except RuntimeError:
@@ -190,8 +190,10 @@ python_LIBPL = %r
INCLUDEPY = %r
Py_ENABLE_SHARED = %r
MACOSX_DEPLOYMENT_TARGET = %r
+LDVERSION = %r
""" % (python, python_prefix, python_SO, python_SYSLIBS, python_LDFLAGS, python_SHLIBS,
- python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED, python_MACOSX_DEPLOYMENT_TARGET))
+ python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED, python_MACOSX_DEPLOYMENT_TARGET,
+ python_LDVERSION))
# Allow some python overrides from env vars for cross-compiling
os_env = dict(os.environ)
@@ -230,7 +232,9 @@ MACOSX_DEPLOYMENT_TARGET = %r
parse_flags(python_LDFLAGS, 'PYEMBED', env)
result = False
- name = 'python' + env['PYTHON_VERSION']
+ if not python_LDVERSION:
+ python_LDVERSION = env['PYTHON_VERSION']
+ name = 'python' + python_LDVERSION
if python_LIBDIR is not None:
path = [python_LIBDIR]
@@ -245,7 +249,7 @@ MACOSX_DEPLOYMENT_TARGET = %r
if not result:
conf.log.write("\n\n# try again with -L$prefix/libs, and pythonXY name rather than pythonX.Y (win32)\n")
path = [os.path.join(python_prefix, "libs")]
- name = 'python' + env['PYTHON_VERSION'].replace('.', '')
+ name = 'python' + python_LDVERSION.replace('.', '')
result = conf.check(lib=name, uselib='PYEMBED', libpath=path)
if result:
--
2.1.0
From 92f2045360560f0a921ab0b2ed20f55c07e4053d Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 12 Nov 2014 19:49:45 +0100
Subject: [PATCH 02/13] buildtools: Use all of pyext_PATTERN in
map_shlib_extension
In Python 3, C extension module filenames have an ABI tag;
the pyext_PATTERN is e.g. "%s.cpython-34m.so".
The build system was only using the last dot-separated element
of that extension (the ".so").
Use the whole extension when constructing the final filename.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_utils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index df4a552..e8bc0f3 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -577,7 +577,7 @@ def map_shlib_extension(ctx, name, python=False):
return name
(root1, ext1) = os.path.splitext(name)
if python:
- (root2, ext2) = os.path.splitext(ctx.env.pyext_PATTERN)
+ return ctx.env.pyext_PATTERN % root1
else:
(root2, ext2) = os.path.splitext(ctx.env.shlib_PATTERN)
return root1+ext2
--
2.1.0
From 78a336aca541701ae0b96a13d3295288d38c3139 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 16 Jan 2015 13:02:37 +0100
Subject: [PATCH 03/13] buildtools: Use 'pyfeature' instead of pyembed and
pyext arguments
This will allow defining other Python-related features, which will be
passed through the stack the same way as the existing two.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 2 +-
buildtools/wafsamba/wafsamba.py | 37 +++++++++++++++++--------------------
lib/ldb/wscript | 2 +-
lib/talloc/wscript | 2 +-
python/wscript_build | 4 ++--
source4/lib/policy/wscript_build | 2 +-
source4/librpc/wscript_build | 2 +-
source4/param/wscript_build | 4 ++--
source4/smbd/wscript_build | 2 +-
source4/torture/wscript_build | 2 +-
source4/utils/wscript_build | 2 +-
source4/web_server/wscript_build | 4 ++--
testsuite/headers/wscript_build | 2 +-
13 files changed, 32 insertions(+), 35 deletions(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index a371b43..f53ac90 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -60,7 +60,7 @@ def SAMBA_PYTHON(bld, name,
vars=vars,
realname=realname,
link_name=link_name,
- pyext=True,
+ pyfeature='pyext',
target_type='PYTHON',
install_path='${PYTHONARCHDIR}',
allow_undefined_symbols=True,
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index c054315..1beb026 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -126,8 +126,7 @@ def SAMBA_LIBRARY(bld, libname, source,
subdir=None,
install_path=None,
install=True,
- pyembed=False,
- pyext=False,
+ pyfeature=None,
target_type='LIBRARY',
bundled_extension=False,
bundled_name=None,
@@ -193,8 +192,7 @@ def SAMBA_LIBRARY(bld, libname, source,
depends_on = depends_on,
hide_symbols = hide_symbols,
allow_warnings = allow_warnings,
- pyembed = pyembed,
- pyext = pyext,
+ pyfeature = pyfeature,
local_include = local_include,
global_include = global_include)
@@ -241,10 +239,8 @@ def SAMBA_LIBRARY(bld, libname, source,
ldflags = TO_LIST(ldflags)
features = 'cc cshlib symlink_lib install_lib'
- if pyext:
- features += ' pyext'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
if abi_directory:
features += ' abi_check'
@@ -336,7 +332,7 @@ def SAMBA_BINARY(bld, binname, source,
local_include=True,
global_include=True,
subsystem_name=None,
- pyembed=False,
+ pyfeature=None,
vars=None,
subdir=None,
install=True,
@@ -352,8 +348,8 @@ def SAMBA_BINARY(bld, binname, source,
return
features = 'cc cprogram symlink_bin install_bin'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
obj_target = binname + '.objlist'
@@ -379,6 +375,10 @@ def SAMBA_BINARY(bld, binname, source,
# first create a target for building the object files for this binary
# by separating in this way, we avoid recompiling the C files
# separately for the install binary and the build binary
+ if pyfeature == 'pyembed':
+ subsystem_pyfeature = 'pyext'
+ else:
+ subsystem_pyfeature = None
bld.SAMBA_SUBSYSTEM(obj_target,
source = source,
deps = deps,
@@ -390,7 +390,7 @@ def SAMBA_BINARY(bld, binname, source,
local_include = local_include,
global_include = global_include,
use_hostcc = use_hostcc,
- pyext = pyembed,
+ pyfeature = subsystem_pyfeature,
use_global_deps= use_global_deps)
bld.SET_BUILD_GROUP(group)
@@ -438,7 +438,7 @@ def SAMBA_MODULE(bld, modname, source,
vars=None,
subdir=None,
enabled=True,
- pyembed=False,
+ pyfeature=None,
manpages=None,
allow_undefined_symbols=False,
allow_warnings=False
@@ -508,7 +508,7 @@ def SAMBA_MODULE(bld, modname, source,
bundled_name=build_name,
link_name=build_link_name,
install_path="${MODULESDIR}/%s" % subsystem,
- pyembed=pyembed,
+ pyfeature=pyfeature,
manpages=manpages,
allow_undefined_symbols=allow_undefined_symbols,
allow_warnings=allow_warnings
@@ -544,8 +544,7 @@ def SAMBA_SUBSYSTEM(bld, modname, source,
subdir=None,
hide_symbols=False,
allow_warnings=False,
- pyext=False,
- pyembed=False):
+ pyfeature=None):
'''define a Samba subsystem'''
if not enabled:
@@ -576,10 +575,8 @@ def SAMBA_SUBSYSTEM(bld, modname, source,
bld.SET_BUILD_GROUP(group)
features = 'cc'
- if pyext:
- features += ' pyext'
- if pyembed:
- features += ' pyembed'
+ if pyfeature:
+ features += ' ' + pyfeature
t = bld(
features = features,
diff --git a/lib/ldb/wscript b/lib/ldb/wscript
index 1067a00..47b8c46 100755
--- a/lib/ldb/wscript
+++ b/lib/ldb/wscript
@@ -129,7 +129,7 @@ def build(bld):
vnum=VERSION,
private_library=private_library,
pc_files='pyldb-util.pc',
- pyembed=True,
+ pyfeature='pyembed',
abi_directory='ABI',
abi_match='pyldb_*')
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 3446146..025cddf 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -116,7 +116,7 @@ def build(bld):
bld.SAMBA_LIBRARY('pytalloc-util',
source='pytalloc_util.c',
public_deps='talloc',
- pyembed=True,
+ pyfeature='pyembed',
vnum=VERSION,
hide_symbols=True,
abi_directory='ABI',
diff --git a/python/wscript_build b/python/wscript_build
index a40b583..56d671e 100644
--- a/python/wscript_build
+++ b/python/wscript_build
@@ -5,14 +5,14 @@ bld.SAMBA_LIBRARY('samba_python',
deps='LIBPYTHON pytalloc-util pyrpc_util',
grouping_library=True,
private_library=True,
- pyembed=True)
+ pyfeature='pyembed')
bld.SAMBA_SUBSYSTEM('LIBPYTHON',
source='modules.c',
public_deps='',
init_function_sentinel='{NULL,NULL}',
deps='talloc',
- pyext=True,
+ pyfeature='pyext',
)
diff --git a/source4/lib/policy/wscript_build b/source4/lib/policy/wscript_build
index b8ba638..8d5ea59 100644
--- a/source4/lib/policy/wscript_build
+++ b/source4/lib/policy/wscript_build
@@ -5,7 +5,7 @@ bld.SAMBA_LIBRARY('samba-policy',
pc_files='samba-policy.pc',
public_deps='ldb samba-net',
vnum='0.0.1',
- pyembed=True,
+ pyfeature='pyembed',
public_headers='policy.h'
)
diff --git a/source4/librpc/wscript_build b/source4/librpc/wscript_build
index 9b96437..d5d9b06 100755
--- a/source4/librpc/wscript_build
+++ b/source4/librpc/wscript_build
@@ -143,7 +143,7 @@ bld.SAMBA_LIBRARY('dcerpc',
bld.SAMBA_SUBSYSTEM('pyrpc_util',
source='rpc/pyrpc_util.c',
public_deps='pytalloc-util pyparam_util dcerpc MESSAGING',
- pyext=True,
+ pyfeature='pyext',
)
diff --git a/source4/param/wscript_build b/source4/param/wscript_build
index 4585a83..87f2eee 100644
--- a/source4/param/wscript_build
+++ b/source4/param/wscript_build
@@ -3,7 +3,7 @@
bld.SAMBA_SUBSYSTEM('PROVISION',
source='provision.c pyparam.c',
deps='LIBPYTHON pyparam_util ldb pytalloc-util pyldb-util',
- pyext=True,
+ pyfeature='pyext',
)
@@ -50,7 +50,7 @@ bld.SAMBA_SUBSYSTEM('param_options',
bld.SAMBA_SUBSYSTEM('pyparam_util',
source='pyparam_util.c',
deps='LIBPYTHON samba-hostconfig',
- pyext=True,
+ pyfeature='pyext',
)
bld.SAMBA_LIBRARY('shares',
diff --git a/source4/smbd/wscript_build b/source4/smbd/wscript_build
index 12d4e8b..42e442c 100644
--- a/source4/smbd/wscript_build
+++ b/source4/smbd/wscript_build
@@ -22,7 +22,7 @@ bld.SAMBA_BINARY('samba',
subsystem_name='service',
deps='''events process_model service samba-hostconfig samba-util POPT_SAMBA
popt gensec registry ntptr ntvfs share cluster COMMON_SCHANNEL SECRETS''',
- pyembed=True,
+ pyfeature='pyembed',
install_path='${SBINDIR}',
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
diff --git a/source4/torture/wscript_build b/source4/torture/wscript_build
index bbb9e9a..7f746ca 100755
--- a/source4/torture/wscript_build
+++ b/source4/torture/wscript_build
@@ -172,7 +172,7 @@ bld.SAMBA_BINARY('smbtorture',
manpages='man/smbtorture.1',
public_headers='smbtorture.h',
deps='torturemain torture popt POPT_SAMBA POPT_CREDENTIALS dcerpc LIBCLI_SMB SMBREADLINE ' + TORTURE_MODULES,
- pyembed=True
+ pyfeature='pyembed',
)
bld.SAMBA_BINARY('gentest',
diff --git a/source4/utils/wscript_build b/source4/utils/wscript_build
index 046e237..cd12da3 100644
--- a/source4/utils/wscript_build
+++ b/source4/utils/wscript_build
@@ -6,7 +6,7 @@ bld.SAMBA_BINARY('ntlm_auth4',
deps='''samba-hostconfig samba-util popt
POPT_SAMBA POPT_CREDENTIALS gensec LIBCLI_RESOLVE
auth4 NTLMSSP_COMMON MESSAGING events service''',
- pyembed=True,
+ pyfeature='pyembed',
install=False
)
diff --git a/source4/web_server/wscript_build b/source4/web_server/wscript_build
index b845067..1c44785 100644
--- a/source4/web_server/wscript_build
+++ b/source4/web_server/wscript_build
@@ -3,7 +3,7 @@
bld.SAMBA_SUBSYSTEM('WEB_WSGI',
source='wsgi.c',
- pyext=True,
+ pyfeature='pyext',
deps='talloc LIBTSOCKET',
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
@@ -14,7 +14,7 @@ bld.SAMBA_MODULE('service_web',
subsystem='service',
init_function='server_service_web_init',
deps='LIBTLS process_model LIBPYTHON WEB_WSGI',
- pyembed=True,
+ pyfeature='pyembed',
internal_module=False,
enabled=bld.AD_DC_BUILD_IS_ENABLED()
)
diff --git a/testsuite/headers/wscript_build b/testsuite/headers/wscript_build
index f612ad9..f201b05 100644
--- a/testsuite/headers/wscript_build
+++ b/testsuite/headers/wscript_build
@@ -41,7 +41,7 @@ if bld.env.DEVELOPER_MODE:
bld.SAMBA_BINARY('test_headers',
source='test_headers.c',
includes="#include/public",
- pyembed=True,
+ pyfeature='pyembed',
cflags=cflags,
local_include=True,
global_include=False,
--
2.1.0
From 22a667eb99c8d55c978b52ae898b243e0ce17b6e Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 14:22:22 +0100
Subject: [PATCH 04/13] buildtools: Add --extra-python configure option
This will allow building Python support for two different Python versions
at the same time.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 4 ++++
buildtools/wafsamba/wscript | 8 ++++++++
2 files changed, 12 insertions(+)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index f53ac90..831a5a7 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -20,6 +20,10 @@ def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
conf.check_python_headers(mandatory)
conf.env["python_headers_checked"] = "yes"
+
+ if conf.env['EXTRAPYTHON_VERSION'] == conf.env['PYTHON_VERSION']:
+ raise Utils.WafError("extrapython %s is same as main python %s" % (
+ conf.env['EXTRAPYTHON_VERSION'], conf.env['PYTHON_VERSION']))
else:
conf.msg("python headers", "using cache")
diff --git a/buildtools/wafsamba/wscript b/buildtools/wafsamba/wscript
index 8027c00..79cf51f 100755
--- a/buildtools/wafsamba/wscript
+++ b/buildtools/wafsamba/wscript
@@ -195,6 +195,12 @@ def set_options(opt):
help='tag release in git at the same time',
type='string', action='store', dest='TAG_RELEASE')
+ opt.add_option('--extra-python', type=str,
+ help=("build selected libraries for the specified "
+ "additional version of Python "
+ "(example: --extra-python=/usr/bin/python3)"),
+ metavar="PYTHON", dest='EXTRA_PYTHON', default=None)
+
@wafsamba.runonce
def configure(conf):
@@ -266,6 +272,8 @@ def configure(conf):
conf.env.AUTOCONF_HOST = Options.options.AUTOCONF_HOST
conf.env.AUTOCONF_PROGRAM_PREFIX = Options.options.AUTOCONF_PROGRAM_PREFIX
+ conf.env.EXTRA_PYTHON = Options.options.EXTRA_PYTHON
+
if (conf.env.AUTOCONF_HOST and
conf.env.AUTOCONF_BUILD and
conf.env.AUTOCONF_BUILD != conf.env.AUTOCONF_HOST):
--
2.1.0
From acda9dc5ea5b8586117fbc7c928f0b8d1a950134 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 17:25:07 +0100
Subject: [PATCH 05/13] buildtools: Store separate configuration for extra
python
For extrapython, we store config in a separate set of env variables.
During the configure step we swap these with the normal *PYTHON* equivalents,
use waf's regular Python tool, and then switch them all back.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 88 ++++++++++++++++++++++++++++++++++++-
1 file changed, 87 insertions(+), 1 deletion(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 831a5a7..15ebf93 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -1,23 +1,109 @@
-# waf build tool for building IDL files with pidl
+import contextlib
import Build
from samba_utils import *
from samba_autoconf import *
+from Logs import warn
from Configure import conf
+# For extrapython, we store config in a separate set of env variables.
+# During the configure step we swap these with the normal *PYTHON* equivalents,
+# use waf's regular Python tool, and then switch them all back.
+EXTRAPYTHON_ENV_KEYS = (
+ ('PYTHON', 'EXTRAPYTHON'),
+ ('PYTHONDIR', 'EXTRAPYTHONDIR'),
+ ('PYTHONARCHDIR', 'EXTRAPYTHONARCHDIR'),
+ ('PYTHON_VERSION', 'EXTRAPYTHON_VERSION'),
+
+ ('CCFLAGS_PYEXT', 'CCFLAGS_EXTRAPYEXT'),
+ ('CCFLAGS_PYEMBED', 'CCFLAGS_EXTRAPYEMBED'),
+
+ ('LIBPATH_PYEXT', 'LIBPATH_EXTRAPYEXT'),
+ ('LIBPATH_PYEMBED', 'LIBPATH_EXTRAPYEMBED'),
+
+ ('LIB_PYEXT', 'LIB_EXTRAPYEXT'),
+ ('LIB_PYEMBED', 'LIB_EXTRAPYEMBED'),
+
+ ('CPPPATH_PYEXT', 'CPPPATH_EXTRAPYEXT'),
+ ('CPPPATH_PYEMBED', 'CPPPATH_EXTRAPYEMBED'),
+
+ ('LINKFLAGS_PYEXT', 'LINKFLAGS_EXTRAPYEXT'),
+ ('LINKFLAGS_PYEMBED', 'LINKFLAGS_EXTRAPYEMBED'),
+
+ ('HAVE_PYTHON_H', 'EXTRAPY_HAVE_PYTHON_H'),
+
+ ('PYTHON_CONFIG', 'EXTRAPYTHON_CONFIG'),
+
+ ('pyext_PATTERN', 'extrapyext_PATTERN'),
+ ('PYCMD', 'EXTRAPYCMD'),
+ ('PYFLAGS', 'EXTRAPYFLAGS'),
+ ('PYFLAGS_OPT', 'EXTRAPYFLAGS_OPT'),
+)
+EXTRAPYTHON_DEFINES_KEYS = (
+ ('PYTHONDIR', 'EXTRAPYTHONDIR'),
+ ('PYTHONARCHDIR', 'EXTRAPYTHONARCHDIR'),
+ ('HAVE_PYTHON_H', 'EXTRAPY_HAVE_PYTHON_H'),
+)
+
+
+ at contextlib.contextmanager
+def extrapython_map(env, keys):
+ saved = {}
+ for key, extrakey in keys:
+ if key in env:
+ saved[key] = env[key]
+ if extrakey in env:
+ env[key] = env[extrakey]
+ else:
+ env.pop(key, None)
+ yield
+ for key, extrakey in keys:
+ if key in env:
+ env[extrakey] = env[key]
+ else:
+ env.pop(extrakey, None)
+ if key in saved:
+ env[key] = saved[key]
+ else:
+ env.pop(key, None)
+
+
+ at contextlib.contextmanager
+def extrapython_env(env):
+ with extrapython_map(env, EXTRAPYTHON_ENV_KEYS):
+ with extrapython_map(env['defines'], EXTRAPYTHON_DEFINES_KEYS):
+ yield
+
+
@conf
def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
# enable tool to build python extensions
+ if conf.env['EXTRA_PYTHON']:
+ with extrapython_env(conf.env):
+ conf.env['PYTHON'] = conf.env['EXTRA_PYTHON']
+ conf.find_program('python', var='PYTHON', mandatory=True)
+ conf.check_tool('python')
+ try:
+ conf.check_python_version((3, 3, 0))
+ except Exception:
+ warn('extra-python needs to be Python 3.3 or later')
+ raise
+
conf.find_program('python', var='PYTHON', mandatory=mandatory)
conf.check_tool('python')
path_python = conf.find_program('python')
conf.env.PYTHON_SPECIFIED = (conf.env.PYTHON != path_python)
conf.check_python_version(version)
+
@conf
def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
+ pre_dict = conf.env.get_merged_dict()
+ if conf.env['EXTRA_PYTHON']:
+ with extrapython_env(conf.env):
+ conf.check_python_headers(mandatory=True)
conf.check_python_headers(mandatory)
conf.env["python_headers_checked"] = "yes"
--
2.1.0
From 6a434140999b8cafa163bf553f5ff9ea79525473 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 16 Jan 2015 14:52:26 +0100
Subject: [PATCH 06/13] buildtools: Add features and type for extra-python
support
This allows support for an additional version of Python to be built.
Two additional features (extrapyext & extrapyembed), and a target type
(EXTRAPYTHON) are added.
They work the same way as the "non-extra" variants, but use configuration
specific to the extra-python version.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_deps.py | 28 +++++++++++-----------
buildtools/wafsamba/samba_install.py | 8 ++++---
buildtools/wafsamba/samba_python.py | 45 ++++++++++++++++++++++++++++++++----
buildtools/wafsamba/samba_utils.py | 17 +++++++++-----
buildtools/wafsamba/stale_files.py | 2 ++
buildtools/wafsamba/symbols.py | 19 +++++++++++----
buildtools/wafsamba/wafsamba.py | 11 +++++----
7 files changed, 95 insertions(+), 35 deletions(-)
diff --git a/buildtools/wafsamba/samba_deps.py b/buildtools/wafsamba/samba_deps.py
index 3be9956..6948d65 100644
--- a/buildtools/wafsamba/samba_deps.py
+++ b/buildtools/wafsamba/samba_deps.py
@@ -72,7 +72,7 @@ def build_dependencies(self):
the full dependency list for a target until we have all of the targets declared.
'''
- if self.samba_type in ['LIBRARY', 'BINARY', 'PYTHON']:
+ if self.samba_type in ['LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON']:
self.uselib = list(self.final_syslibs)
self.uselib_local = list(self.final_libs)
self.add_objects = list(self.final_objects)
@@ -279,7 +279,7 @@ def check_duplicate_sources(bld, tgt_list):
# build a list of targets that each source file is part of
for t in tgt_list:
sources = []
- if not targets[t.sname] in [ 'LIBRARY', 'BINARY', 'PYTHON' ]:
+ if not targets[t.sname] in [ 'LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON' ]:
continue
for obj in t.add_objects:
t2 = t.bld.name_to_obj(obj, bld.env)
@@ -312,7 +312,7 @@ def check_orphaned_targets(bld, tgt_list):
if getattr(t, 'samba_used', False):
continue
type = target_dict[t.sname]
- if not type in ['BINARY', 'LIBRARY', 'MODULE', 'ET', 'PYTHON']:
+ if not type in ['BINARY', 'LIBRARY', 'MODULE', 'ET', 'PYTHON', 'EXTRAPYTHON']:
if re.search('^PIDL_', t.sname) is None:
Logs.warn("Target %s of type %s is unused by any other target" % (t.sname, type))
@@ -366,7 +366,7 @@ def show_final_deps(bld, tgt_list):
targets = LOCAL_CACHE(bld, 'TARGET_TYPE')
for t in tgt_list:
- if not targets[t.sname] in ['LIBRARY', 'BINARY', 'PYTHON', 'SUBSYSTEM']:
+ if not targets[t.sname] in ['LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON', 'SUBSYSTEM']:
continue
debug('deps: final dependencies for target %s: uselib=%s uselib_local=%s add_objects=%s',
t.sname, t.uselib, getattr(t, 'uselib_local', []), getattr(t, 'add_objects', []))
@@ -449,7 +449,9 @@ def build_direct_deps(bld, tgt_list):
sys.exit(1)
if targets[d] in [ 'EMPTY', 'DISABLED' ]:
continue
- if targets[d] == 'PYTHON' and targets[t.sname] != 'PYTHON' and t.sname.find('.objlist') == -1:
+ if (targets[d] in ('PYTHON', 'EXTRAPYTHON') and
+ targets[t.sname] not in ('PYTHON', 'EXTRAPYTHON') and
+ t.sname.find('.objlist') == -1):
# this check should be more restrictive, but for now we have pidl-generated python
# code that directly depends on other python modules
Logs.error('ERROR: Target %s has dependency on python module %s' % (t.sname, d))
@@ -475,7 +477,7 @@ def build_direct_deps(bld, tgt_list):
sys.exit(1)
if t2.samba_type in [ 'LIBRARY', 'MODULE' ]:
t.direct_libs.add(d)
- elif t2.samba_type in [ 'SUBSYSTEM', 'ASN1', 'PYTHON' ]:
+ elif t2.samba_type in [ 'SUBSYSTEM', 'ASN1', 'PYTHON', 'EXTRAPYTHON' ]:
t.direct_objects.add(d)
debug('deps: built direct dependencies')
@@ -669,7 +671,7 @@ def break_dependency_loops(bld, tgt_list):
if t.samba_type in ['SUBSYSTEM']:
loops[loop] = loops[loop].union(t.indirect_objects)
loops[loop] = loops[loop].union(t.direct_objects)
- if t.samba_type in ['LIBRARY','PYTHON']:
+ if t.samba_type in ['LIBRARY','PYTHON','EXTRAPYTHON']:
loops[loop] = loops[loop].union(t.indirect_libs)
loops[loop] = loops[loop].union(t.direct_libs)
if loop in loops[loop]:
@@ -717,7 +719,7 @@ def reduce_objects(bld, tgt_list):
changed = False
- for type in ['BINARY', 'PYTHON', 'LIBRARY']:
+ for type in ['BINARY', 'PYTHON', 'EXTRAPYTHON', 'LIBRARY']:
for t in tgt_list:
if t.samba_type != type: continue
# if we will indirectly link to a target then we don't need it
@@ -817,7 +819,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# find any library loops
for t in tgt_list:
- if t.samba_type in ['LIBRARY', 'PYTHON']:
+ if t.samba_type in ['LIBRARY', 'PYTHON', 'EXTRAPYTHON']:
for l in t.final_libs.copy():
t2 = bld.name_to_obj(l, bld.env)
if t.sname in t2.final_libs:
@@ -841,7 +843,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# we now need to make corrections for any library loops we broke up
# any target that depended on the target of the loop and doesn't
# depend on the source of the loop needs to get the loop source added
- for type in ['BINARY','PYTHON','LIBRARY','BINARY']:
+ for type in ['BINARY','PYTHON','EXTRAPYTHON','LIBRARY','BINARY']:
for t in tgt_list:
if t.samba_type != type: continue
for loop in loops:
@@ -874,7 +876,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# add in any syslib dependencies
for t in tgt_list:
- if not t.samba_type in ['BINARY','PYTHON','LIBRARY','SUBSYSTEM']:
+ if not t.samba_type in ['BINARY','PYTHON','EXTRAPYTHON','LIBRARY','SUBSYSTEM']:
continue
syslibs = set()
for d in t.final_objects:
@@ -891,7 +893,7 @@ def calculate_final_deps(bld, tgt_list, loops):
# find any unresolved library loops
lib_loop_error = False
for t in tgt_list:
- if t.samba_type in ['LIBRARY', 'PYTHON']:
+ if t.samba_type in ['LIBRARY', 'PYTHON', 'EXTRAPYTHON']:
for l in t.final_libs.copy():
t2 = bld.name_to_obj(l, bld.env)
if t.sname in t2.final_libs:
@@ -935,7 +937,7 @@ def show_object_duplicates(bld, tgt_list):
Logs.info("showing duplicate objects")
for t in tgt_list:
- if not targets[t.sname] in [ 'LIBRARY', 'PYTHON' ]:
+ if not targets[t.sname] in [ 'LIBRARY', 'PYTHON', 'EXTRAPYTHON' ]:
continue
for n in getattr(t, 'final_objects', set()):
t2 = bld.name_to_obj(n, bld.env)
diff --git a/buildtools/wafsamba/samba_install.py b/buildtools/wafsamba/samba_install.py
index aa7f143..4a5780b 100644
--- a/buildtools/wafsamba/samba_install.py
+++ b/buildtools/wafsamba/samba_install.py
@@ -105,10 +105,12 @@ def install_library(self):
install_link = None
if getattr(self, 'soname', ''):
install_link = self.soname
- if getattr(self, 'samba_type', None) == 'PYTHON':
- inst_name = bld.make_libname(t.target, nolibprefix=True, python=True)
+ samba_type = getattr(self, 'samba_type', None)
+ if samba_type in ('PYTHON', 'EXTRAPYTHON'):
+ inst_name = bld.make_libname(t.target, nolibprefix=True,
+ target_type=samba_type)
else:
- inst_name = bld.make_libname(t.target)
+ inst_name = bld.make_libname(t.target)
elif self.vnum:
vnum_base = self.vnum.split('.')[0]
install_name = bld.make_libname(target_name, version=self.vnum)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 15ebf93..e6c0c6f 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -4,9 +4,37 @@ import Build
from samba_utils import *
from samba_autoconf import *
from Logs import warn
+from TaskGen import extension, before, after, feature
from Configure import conf
+
+ at feature('extrapyext')
+ at before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars', 'apply_bundle')
+ at after('vars_target_cshlib')
+def init_extrapyext(self):
+ self.default_install_path = '${PYTHONARCHDIR}'
+ self.uselib = self.to_list(getattr(self, 'uselib', ''))
+ if not 'EXTRAPYEXT' in self.uselib:
+ self.uselib.append('EXTRAPYEXT')
+
+
+ at before('apply_link', 'apply_lib_vars', 'apply_type_vars')
+ at after('apply_bundle')
+ at feature('extrapyext')
+def extrapyext_shlib_ext(self):
+ # override shlib_PATTERN set by the osx module
+ self.env['shlib_PATTERN'] = self.env['extrapyext_PATTERN']
+
+
+ at before('apply_incpaths', 'apply_lib_vars', 'apply_type_vars')
+ at feature('extrapyembed')
+def init_extrapyembed(self):
+ self.uselib = self.to_list(getattr(self, 'uselib', ''))
+ if not 'EXTRAPYEMBED' in self.uselib:
+ self.uselib.append('EXTRAPYEMBED')
+
+
# For extrapython, we store config in a separate set of env variables.
# During the configure step we swap these with the normal *PYTHON* equivalents,
# use waf's regular Python tool, and then switch them all back.
@@ -125,7 +153,8 @@ def SAMBA_PYTHON(bld, name,
local_include=True,
vars=None,
install=True,
- enabled=True):
+ enabled=True,
+ extra_python=False):
'''build a python extension for Samba'''
# when we support static python modules we'll need to gather
@@ -140,6 +169,14 @@ def SAMBA_PYTHON(bld, name,
else:
link_name = None
+ if extra_python:
+ pyfeature = 'extrapyext'
+ target_type = 'EXTRAPYTHON'
+ install_path = '${EXTRAPYTHONARCHDIR}'
+ else:
+ pyfeature = 'pyext'
+ target_type = 'PYTHON'
+ install_path = '${PYTHONARCHDIR}'
bld.SAMBA_LIBRARY(name,
source=source,
deps=deps,
@@ -150,9 +187,9 @@ def SAMBA_PYTHON(bld, name,
vars=vars,
realname=realname,
link_name=link_name,
- pyfeature='pyext',
- target_type='PYTHON',
- install_path='${PYTHONARCHDIR}',
+ pyfeature=pyfeature,
+ target_type=target_type,
+ install_path=install_path,
allow_undefined_symbols=True,
allow_warnings=True,
install=install,
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index e8bc0f3..fd04a49 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -568,7 +568,7 @@ def reconfigure(ctx):
Scripting.check_configured(bld)
-def map_shlib_extension(ctx, name, python=False):
+def map_shlib_extension(ctx, name, target_type=None):
'''map a filename with a shared library extension of .so to the real shlib name'''
if name is None:
return None
@@ -576,8 +576,10 @@ def map_shlib_extension(ctx, name, python=False):
# some libraries have specified versions in the wscript rule
return name
(root1, ext1) = os.path.splitext(name)
- if python:
+ if target_type == 'PYTHON':
return ctx.env.pyext_PATTERN % root1
+ elif target_type == 'EXTRAPYTHON':
+ return ctx.env.extrapyext_PATTERN % root1
else:
(root2, ext2) = os.path.splitext(ctx.env.shlib_PATTERN)
return root1+ext2
@@ -591,15 +593,18 @@ def apply_pattern(filename, pattern):
basename = os.path.basename(filename)
return os.path.join(dirname, pattern % basename)
-def make_libname(ctx, name, nolibprefix=False, version=None, python=False):
+def make_libname(ctx, name, nolibprefix=False, version=None, target_type=None):
"""make a library filename
Options:
nolibprefix: don't include the lib prefix
version : add a version number
- python : if we should use python module name conventions"""
+ target_type: 'PYTHON' or 'EXTRAPYTHON' if we should use python module name conventions
+ """
- if python:
+ if target_type == 'PYTHON':
libname = apply_pattern(name, ctx.env.pyext_PATTERN)
+ elif target_type == 'EXTRAPYTHON':
+ libname = apply_pattern(name, ctx.env.extrapyext_PATTERN)
else:
libname = apply_pattern(name, ctx.env.shlib_PATTERN)
if nolibprefix and libname[0:3] == 'lib':
@@ -626,7 +631,7 @@ def get_tgt_list(bld):
tgt_list = []
for tgt in targets:
type = targets[tgt]
- if not type in ['SUBSYSTEM', 'MODULE', 'BINARY', 'LIBRARY', 'ASN1', 'PYTHON']:
+ if not type in ['SUBSYSTEM', 'MODULE', 'BINARY', 'LIBRARY', 'ASN1', 'PYTHON', 'EXTRAPYTHON']:
continue
t = bld.name_to_obj(tgt, bld.env)
if t is None:
diff --git a/buildtools/wafsamba/stale_files.py b/buildtools/wafsamba/stale_files.py
index 2dd08e1..8c5d5ee 100644
--- a/buildtools/wafsamba/stale_files.py
+++ b/buildtools/wafsamba/stale_files.py
@@ -71,6 +71,8 @@ def replace_refill_task_list(self):
t = samba_utils.apply_pattern(t, bld.env.shlib_PATTERN)
if ttype == 'PYTHON':
t = samba_utils.apply_pattern(t, bld.env.pyext_PATTERN)
+ elif ttype == 'EXTRAPYTHON':
+ t = samba_utils.apply_pattern(t, bld.env.extrapyext_PATTERN)
p = os.path.join(x.path.abspath(bld.env), t)
p = os.path.normpath(p)
expected.append(p)
diff --git a/buildtools/wafsamba/symbols.py b/buildtools/wafsamba/symbols.py
index daa18b9..6750a8a 100644
--- a/buildtools/wafsamba/symbols.py
+++ b/buildtools/wafsamba/symbols.py
@@ -278,7 +278,7 @@ def build_library_dict(bld, tgt_list):
bld.env.library_dict = {}
for t in tgt_list:
- if t.samba_type in [ 'LIBRARY', 'PYTHON' ]:
+ if t.samba_type in [ 'LIBRARY', 'PYTHON', 'EXTRAPYTHON' ]:
linkpath = os.path.realpath(t.link_task.outputs[0].abspath(bld.env))
bld.env.library_dict[linkpath] = t.sname
@@ -293,10 +293,12 @@ def build_syslib_sets(bld, tgt_list):
syslibs = {}
objmap = {}
for t in tgt_list:
- if getattr(t, 'uselib', []) and t.samba_type in [ 'LIBRARY', 'BINARY', 'PYTHON' ]:
+ if getattr(t, 'uselib', []) and t.samba_type in [ 'LIBRARY', 'BINARY', 'PYTHON', 'EXTRAPYTHON' ]:
for lib in t.uselib:
if lib in ['PYEMBED', 'PYEXT']:
lib = "python"
+ elif lib in ['EXTRAPYEMBED', 'EXTRAPYEXT']:
+ lib = "python" + bld.env.EXTRAPYTHON_VERSION
if not lib in syslibs:
syslibs[lib] = []
syslibs[lib].append(t)
@@ -356,7 +358,8 @@ def build_autodeps(bld, t):
if t.in_library == depname:
# no need to depend on the library we are part of
continue
- if depname[0] in ['c', 'python']:
+ if depname[0] in ['c', 'python',
+ 'python' + bld.env.EXTRAPYTHON_VERSION]:
# these don't go into autodeps
continue
if targets[depname[0]] in [ 'SYSLIB' ]:
@@ -484,8 +487,14 @@ def check_syslib_dependencies(bld, t):
features = TO_LIST(t.features)
if 'pyembed' in features or 'pyext' in features:
- if 'python' in bld.env.public_symbols:
- t.unsatisfied_symbols = t.unsatisfied_symbols.difference(bld.env.public_symbols['python'])
+ python_libname = 'python'
+ elif 'extrapyembed' in features or 'extrapyext' in features:
+ python_libname = 'python' + bld.env.EXTRAPYTHON_VERSION
+ else:
+ python_libname = None
+ if python_libname and python_libname in bld.env.public_symbols:
+ py_symbols = bld.env.public_symbols[python_libname]
+ t.unsatisfied_symbols = t.unsatisfied_symbols.difference(py_symbols)
needed = {}
for sym in t.unsatisfied_symbols:
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index 1beb026..29c4f88 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -207,11 +207,12 @@ def SAMBA_LIBRARY(bld, libname, source,
deps = TO_LIST(deps)
deps.append(obj_target)
- realname = bld.map_shlib_extension(realname, python=(target_type=='PYTHON'))
- link_name = bld.map_shlib_extension(link_name, python=(target_type=='PYTHON'))
+ realname = bld.map_shlib_extension(realname, target_type=target_type)
+ link_name = bld.map_shlib_extension(link_name, target_type=target_type)
# we don't want any public libraries without version numbers
- if (not private_library and target_type != 'PYTHON' and not realname):
+ if (not private_library and not realname and
+ target_type not in ('PYTHON', 'EXTRAPYTHON')):
if vnum is None and soname is None:
raise Utils.WafError("public library '%s' must have a vnum" %
libname)
@@ -224,7 +225,7 @@ def SAMBA_LIBRARY(bld, libname, source,
if bundled_name is not None:
pass
- elif target_type == 'PYTHON' or realname or not private_library:
+ elif target_type in ('PYTHON', 'EXTRAPYTHON') or realname or not private_library:
if keep_underscore:
bundled_name = libname
else:
@@ -377,6 +378,8 @@ def SAMBA_BINARY(bld, binname, source,
# separately for the install binary and the build binary
if pyfeature == 'pyembed':
subsystem_pyfeature = 'pyext'
+ elif pyfeature == 'extrapyembed':
+ subsystem_pyfeature = 'extrapyext'
else:
subsystem_pyfeature = None
bld.SAMBA_SUBSYSTEM(obj_target,
--
2.1.0
From 2479fa180a0a1a6037d00ca9f4acbcd78f3bf311 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 23 Jan 2015 16:32:21 +0100
Subject: [PATCH 07/13] buildtools: Don't require pkgconfig file or headers for
extrapython
For system configuration one should use the system version of Python,
not extrapython.
Public headers should be the same for all versions, so the ones for
the main Python version should be used.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/wafsamba.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/buildtools/wafsamba/wafsamba.py b/buildtools/wafsamba/wafsamba.py
index 29c4f88..cd301b5 100644
--- a/buildtools/wafsamba/wafsamba.py
+++ b/buildtools/wafsamba/wafsamba.py
@@ -216,10 +216,10 @@ def SAMBA_LIBRARY(bld, libname, source,
if vnum is None and soname is None:
raise Utils.WafError("public library '%s' must have a vnum" %
libname)
- if pc_files is None:
+ if pc_files is None and pyfeature != 'extrapyembed':
raise Utils.WafError("public library '%s' must have pkg-config file" %
libname)
- if public_headers is None:
+ if public_headers is None and pyfeature != 'extrapyembed':
raise Utils.WafError("public library '%s' must have header files" %
libname)
--
2.1.0
From 7e2a0afbbe319ea76dbc56fda3ffba36667eaf81 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Fri, 23 Jan 2015 17:02:55 +0100
Subject: [PATCH 08/13] buildtools: Enable adding ABI flags to Python utilities
This allows shared libraries to be named, for example,
"libpytalloc-util.cpython-34m.so". The ABI flag enables
libraries for several Python versions to co-exist on
a single system.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index e6c0c6f..878df38 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -1,3 +1,4 @@
+import os
import contextlib
import Build
@@ -67,6 +68,8 @@ EXTRAPYTHON_ENV_KEYS = (
('PYCMD', 'EXTRAPYCMD'),
('PYFLAGS', 'EXTRAPYFLAGS'),
('PYFLAGS_OPT', 'EXTRAPYFLAGS_OPT'),
+
+ ('PYTHON_SO_ABI_FLAG', 'EXTRAPYTHON_SO_ABI_FLAG'),
)
EXTRAPYTHON_DEFINES_KEYS = (
('PYTHONDIR', 'EXTRAPYTHONDIR'),
@@ -125,14 +128,24 @@ def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
conf.check_python_version(version)
+def check_python_headers(conf, mandatory):
+ conf.check_python_headers(mandatory=mandatory)
+
+ if conf.env['PYTHON_VERSION'] > '3':
+ abi_pattern = os.path.splitext(conf.env['pyext_PATTERN'])[0]
+ conf.env['PYTHON_SO_ABI_FLAG'] = abi_pattern % ''
+ else:
+ conf.env['PYTHON_SO_ABI_FLAG'] = ''
+
+
@conf
def SAMBA_CHECK_PYTHON_HEADERS(conf, mandatory=True):
if conf.env["python_headers_checked"] == []:
pre_dict = conf.env.get_merged_dict()
if conf.env['EXTRA_PYTHON']:
with extrapython_env(conf.env):
- conf.check_python_headers(mandatory=True)
- conf.check_python_headers(mandatory)
+ check_python_headers(conf, mandatory=True)
+ check_python_headers(conf, mandatory=mandatory)
conf.env["python_headers_checked"] = "yes"
if conf.env['EXTRAPYTHON_VERSION'] == conf.env['PYTHON_VERSION']:
@@ -196,3 +209,14 @@ def SAMBA_PYTHON(bld, name,
enabled=enabled)
Build.BuildContext.SAMBA_PYTHON = SAMBA_PYTHON
+
+
+def pyembed_libname(bld, name, extrapython=False):
+ if extrapython:
+ return name + bld.env['EXTRAPYTHON_SO_ABI_FLAG']
+ else:
+ return name + bld.env['PYTHON_SO_ABI_FLAG']
+
+ return name
+
+Build.BuildContext.pyembed_libname = pyembed_libname
--
2.1.0
From c334f31b19cc275cb7728fd42294b54524433bc1 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Tue, 10 Mar 2015 18:19:14 +0100
Subject: [PATCH 09/13] buildtools: Add a helper for running Python tests
Add the function samba_utils.RUN_PYTHON_TESTS for running a Python
test. When building for multiple Python versions, all are tested.
Also, add the list of configured Python interpreters to build config.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
buildtools/wafsamba/samba_python.py | 6 ++++++
buildtools/wafsamba/samba_utils.py | 16 ++++++++++++++++
2 files changed, 22 insertions(+)
diff --git a/buildtools/wafsamba/samba_python.py b/buildtools/wafsamba/samba_python.py
index 878df38..e07d33f 100644
--- a/buildtools/wafsamba/samba_python.py
+++ b/buildtools/wafsamba/samba_python.py
@@ -110,6 +110,8 @@ def extrapython_env(env):
@conf
def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
# enable tool to build python extensions
+ interpreters = []
+
if conf.env['EXTRA_PYTHON']:
with extrapython_env(conf.env):
conf.env['PYTHON'] = conf.env['EXTRA_PYTHON']
@@ -120,6 +122,7 @@ def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
except Exception:
warn('extra-python needs to be Python 3.3 or later')
raise
+ interpreters.append(conf.env['EXTRA_PYTHON'])
conf.find_program('python', var='PYTHON', mandatory=mandatory)
conf.check_tool('python')
@@ -127,6 +130,9 @@ def SAMBA_CHECK_PYTHON(conf, mandatory=True, version=(2,4,2)):
conf.env.PYTHON_SPECIFIED = (conf.env.PYTHON != path_python)
conf.check_python_version(version)
+ interpreters.append(conf.env['PYTHON'])
+ conf.env.python_interpreters = interpreters
+
def check_python_headers(conf, mandatory):
conf.check_python_headers(mandatory=mandatory)
diff --git a/buildtools/wafsamba/samba_utils.py b/buildtools/wafsamba/samba_utils.py
index fd04a49..4588be9 100644
--- a/buildtools/wafsamba/samba_utils.py
+++ b/buildtools/wafsamba/samba_utils.py
@@ -386,6 +386,22 @@ def RUN_COMMAND(cmd,
return -1
+def RUN_PYTHON_TESTS(testfiles, pythonpath=None):
+ env = LOAD_ENVIRONMENT()
+ if pythonpath is None:
+ pythonpath = os.path.join(Utils.g_module.blddir, 'python')
+ result = 0
+ for interp in env.python_interpreters:
+ for testfile in testfiles:
+ cmd = "PYTHONPATH=%s %s %s" % (pythonpath, interp, testfile)
+ print('Running Python testwith %s: %s' % (interp, testfile))
+ ret = RUN_COMMAND(cmd)
+ if ret:
+ print('Python test failed: %s' % cmd)
+ result = ret
+ return result
+
+
# make sure we have md5. some systems don't have it
try:
from hashlib import md5
--
2.1.0
From 1339ae74564e9af5f496f3fd8fded155adcd643e Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Thu, 15 Jan 2015 14:07:09 +0100
Subject: [PATCH 10/13] talloc build: Use SAMBA_CHECK_PYTHON for finding Python
Previously the code repeated most of SAMBA_CHECK_PYTHON explicitly.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/wscript | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 025cddf..0f21839 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -59,9 +59,7 @@ def configure(conf):
if not conf.env.disable_python:
# also disable if we don't have the python libs installed
- conf.find_program('python', var='PYTHON')
- conf.check_tool('python')
- conf.check_python_version((2,4,2))
+ conf.SAMBA_CHECK_PYTHON(mandatory=False, version=(2,4,2))
conf.SAMBA_CHECK_PYTHON_HEADERS(mandatory=False)
if not conf.env.HAVE_PYTHON_H:
Logs.warn('Disabling pytalloc-util as python devel libs not found')
--
2.1.0
From 45a1be090a4d4d237313540976b8dd95064b19f1 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Tue, 10 Mar 2015 16:35:05 +0100
Subject: [PATCH 11/13] pytalloc: Add a Python 3 compatibility header
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/py3compat.h | 126 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 126 insertions(+)
create mode 100644 lib/talloc/py3compat.h
diff --git a/lib/talloc/py3compat.h b/lib/talloc/py3compat.h
new file mode 100644
index 0000000..028ce75
--- /dev/null
+++ b/lib/talloc/py3compat.h
@@ -0,0 +1,126 @@
+/*
+ Unix SMB/CIFS implementation.
+ Python 2/3 compatibility layer
+
+ Copyright (C) Petr Viktorin 2015
+
+ ** NOTE! The following LGPL license applies to the talloc
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#ifndef _PY3COMPAT_H_
+#define _PY3COMPAT_H_
+#include <Python.h>
+
+#if PY_MAJOR_VERSION >= 3
+
+/***** Python 3 *****/
+
+#define IS_PY3 1
+
+/* Strings */
+
+#define PyStr_FromString PyUnicode_FromString
+#define PyStr_AsString PyUnicode_AsUTF8
+#define PyStr_FromFormat PyUnicode_FromFormat
+#define PyStr_FromStringAndSize PyUnicode_FromStringAndSize
+#define PyStr_FromFormatV PyUnicode_FromFormatV
+#define PyStr_Check PyUnicode_Check
+#define PyStr_AsUTF8AndSize PyUnicode_AsUTF8AndSize
+
+/* Ints */
+
+#define PyInt_FromLong PyLong_FromLong
+#define PyInt_AsLong PyLong_AsLong
+#define PyInt_Check PyLong_Check
+
+/* Module init */
+
+#define MODULE_INIT_FUNC(name) \
+ PyMODINIT_FUNC PyInit_ ## name(void); \
+ PyMODINIT_FUNC PyInit_ ## name(void)
+
+/* Types */
+
+#define Py_TPFLAGS_HAVE_WEAKREFS 0
+#define Py_TPFLAGS_HAVE_ITER 0
+
+#else
+
+/***** Python 2 *****/
+
+#define IS_PY3 0
+
+/* Strings */
+
+#define PyStr_FromString PyString_FromString
+#define PyStr_AsString PyString_AsString
+#define PyStr_FromFormat PyString_FromFormat
+#define PyStr_FromStringAndSize PyString_FromStringAndSize
+#define PyStr_FromFormatV PyString_FromFormatV
+#define PyStr_Check PyString_Check
+
+#define PyStr_AsUTF8AndSize(pystr, sizeptr) \
+ ((*sizeptr=PyString_Size(pystr)), PyString_AsString(pystr))
+
+#define PyBytes_FromStringAndSize PyString_FromStringAndSize
+#define PyBytes_FromString PyString_FromString
+#define PyBytes_Size PyString_Size
+
+/* Module init */
+
+#define PyModuleDef_HEAD_INIT 0
+
+typedef struct PyModuleDef {
+ int m_base;
+ const char* m_name;
+ const char* m_doc;
+ Py_ssize_t m_size;
+ PyMethodDef *m_methods;
+} PyModuleDef;
+
+#define PyModule_Create(def) \
+ Py_InitModule3((def)->m_name, (def)->m_methods, (def)->m_doc)
+
+#define MODULE_INIT_FUNC(name) \
+ static PyObject *PyInit_ ## name(void); \
+ void init ## name(void); \
+ void init ## name(void) { PyInit_ ## name(); } \
+ static PyObject *PyInit_ ## name(void)
+
+
+#endif
+
+/***** Common *****/
+
+/* Rich comparisons */
+
+#ifndef Py_RETURN_NOTIMPLEMENTED
+#define Py_RETURN_NOTIMPLEMENTED \
+ return Py_INCREF(Py_NotImplemented), Py_NotImplemented
+#endif
+
+#define PY_RICHCMP(val1, val2, op) \
+ ((op) == Py_EQ) ? PyBool_FromLong((val1) == (val2)) : \
+ ((op) == Py_NE) ? PyBool_FromLong((val1) != (val2)) : \
+ ((op) == Py_LT) ? PyBool_FromLong((val1) < (val2)) : \
+ ((op) == Py_GT) ? PyBool_FromLong((val1) > (val2)) : \
+ ((op) == Py_LE) ? PyBool_FromLong((val1) <= (val2)) : \
+ ((op) == Py_GE) ? PyBool_FromLong((val1) >= (val2)) : \
+ (Py_INCREF(Py_NotImplemented), Py_NotImplemented)
+
+#endif // _PY3COMPAT_H_
--
2.1.0
From 2af92d5143a6525d8ec1e2d5cb505b3f8a3d610f Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Tue, 10 Mar 2015 16:40:48 +0100
Subject: [PATCH 12/13] pytalloc: Port to Python 3
- Use native string for repr
- Use rich comparison
Removes the deprecated tp_compare in favor of tp_richcompare.
Disparate types cannot be compared (except for == and !=),
and True or False objects are returned explicitly.
- Use Py_TYPE instead of ob_type
This changed to conform to C aliasing rules,
see http://legacy.python.org/dev/peps/pep-3123/
- Don't provide CObject creation function
A PyCapsule based replacement would be possible,
but might not be necessary considering the function is
not used much.
- Use new-style module initialization
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/pytalloc.c | 40 ++++++++++++++++++++++++----------------
lib/talloc/pytalloc_util.c | 5 +++++
lib/talloc/test_pytalloc.c | 21 +++++++++++++++------
lib/talloc/test_pytalloc.py | 20 +++++++++++++-------
4 files changed, 57 insertions(+), 29 deletions(-)
diff --git a/lib/talloc/pytalloc.c b/lib/talloc/pytalloc.c
index ac4fe0f..68143ca 100644
--- a/lib/talloc/pytalloc.c
+++ b/lib/talloc/pytalloc.c
@@ -20,8 +20,9 @@
#include <Python.h>
#include <talloc.h>
#include <pytalloc.h>
+#include <py3compat.h>
-void inittalloc(void);
+static PyTypeObject TallocObject_Type;
/* print a talloc tree report for a talloc python object */
static PyObject *pytalloc_report_full(PyObject *self, PyObject *args)
@@ -79,8 +80,8 @@ static PyObject *pytalloc_default_repr(PyObject *obj)
pytalloc_Object *talloc_obj = (pytalloc_Object *)obj;
PyTypeObject *type = (PyTypeObject*)PyObject_Type(obj);
- return PyString_FromFormat("<%s talloc object at 0x%p>",
- type->tp_name, talloc_obj->ptr);
+ return PyStr_FromFormat("<%s talloc object at 0x%p>",
+ type->tp_name, talloc_obj->ptr);
}
/**
@@ -97,14 +98,13 @@ static void pytalloc_dealloc(PyObject* self)
/**
* Default (but only slightly more useful than the default) implementation of cmp.
*/
-static int pytalloc_default_cmp(PyObject *_obj1, PyObject *_obj2)
+static PyObject *pytalloc_default_richcmp(PyObject *obj1, PyObject *obj2, int op)
{
- pytalloc_Object *obj1 = (pytalloc_Object *)_obj1,
- *obj2 = (pytalloc_Object *)_obj2;
- if (obj1->ob_type != obj2->ob_type)
- return ((char *)obj1->ob_type - (char *)obj2->ob_type);
-
- return ((char *)pytalloc_get_ptr(obj1) - (char *)pytalloc_get_ptr(obj2));
+ if (PyObject_TypeCheck(obj1, &TallocObject_Type) &&
+ PyObject_TypeCheck(obj2, &TallocObject_Type)) {
+ PY_RICHCMP(pytalloc_get_ptr(obj1), pytalloc_get_ptr(obj2), op);
+ }
+ Py_RETURN_NOTIMPLEMENTED;
}
static PyTypeObject TallocObject_Type = {
@@ -114,21 +114,29 @@ static PyTypeObject TallocObject_Type = {
.tp_dealloc = (destructor)pytalloc_dealloc,
.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
.tp_repr = pytalloc_default_repr,
- .tp_compare = pytalloc_default_cmp,
+ .tp_richcompare = pytalloc_default_richcmp,
+};
+
+static struct PyModuleDef moduledef = {
+ PyModuleDef_HEAD_INIT,
+ .m_name = "talloc",
+ .m_doc = PyDoc_STR("Python wrapping of talloc-maintained objects."),
+ .m_size = -1,
+ .m_methods = talloc_methods,
};
-void inittalloc(void)
+MODULE_INIT_FUNC(talloc)
{
PyObject *m;
if (PyType_Ready(&TallocObject_Type) < 0)
- return;
+ return NULL;
- m = Py_InitModule3("talloc", talloc_methods,
- "Python wrapping of talloc-maintained objects.");
+ m = PyModule_Create(&moduledef);
if (m == NULL)
- return;
+ return NULL;
Py_INCREF(&TallocObject_Type);
PyModule_AddObject(m, "Object", (PyObject *)&TallocObject_Type);
+ return m;
}
diff --git a/lib/talloc/pytalloc_util.c b/lib/talloc/pytalloc_util.c
index 89a093b..ba26a56 100644
--- a/lib/talloc/pytalloc_util.c
+++ b/lib/talloc/pytalloc_util.c
@@ -22,6 +22,7 @@
#include <talloc.h>
#include "pytalloc.h"
#include <assert.h>
+#include <py3compat.h>
_PUBLIC_ PyTypeObject *pytalloc_GetObjectType(void)
{
@@ -97,6 +98,8 @@ _PUBLIC_ PyObject *pytalloc_reference_ex(PyTypeObject *py_type, TALLOC_CTX *mem_
return (PyObject *)ret;
}
+#if !IS_PY3
+
static void py_cobject_talloc_free(void *ptr)
{
talloc_free(ptr);
@@ -110,6 +113,8 @@ _PUBLIC_ PyObject *pytalloc_CObject_FromTallocPtr(void *ptr)
return PyCObject_FromVoidPtr(ptr, py_cobject_talloc_free);
}
+#endif
+
_PUBLIC_ int pytalloc_Check(PyObject *obj)
{
PyTypeObject *tp = pytalloc_GetObjectType();
diff --git a/lib/talloc/test_pytalloc.c b/lib/talloc/test_pytalloc.c
index 2d8372b..215aa8c 100644
--- a/lib/talloc/test_pytalloc.c
+++ b/lib/talloc/test_pytalloc.c
@@ -21,6 +21,7 @@
#include <Python.h>
#include <talloc.h>
#include <pytalloc.h>
+#include <py3compat.h>
static PyObject *testpytalloc_new(PyTypeObject *mod)
{
@@ -96,25 +97,33 @@ static PyTypeObject DObject_Type = {
.tp_doc = "test talloc object that calls a function when underlying data is freed\n",
};
-#define MODULE_DOC "Test utility module for pytalloc"
-void init_test_pytalloc(void);
-void init_test_pytalloc(void)
+static struct PyModuleDef moduledef = {
+ PyModuleDef_HEAD_INIT,
+ .m_name = "_test_pytalloc",
+ .m_doc = PyDoc_STR("Test utility module for pytalloc"),
+ .m_size = -1,
+ .m_methods = test_talloc_methods,
+};
+
+MODULE_INIT_FUNC(_test_pytalloc)
{
PyObject *m;
DObject_Type.tp_base = pytalloc_GetObjectType();
if (PyType_Ready(&DObject_Type) < 0) {
- return;
+ return NULL;
}
- m = Py_InitModule3("_test_pytalloc", test_talloc_methods, MODULE_DOC);
+ m = PyModule_Create(&moduledef);
if (m == NULL) {
- return;
+ return NULL;
}
Py_INCREF(&DObject_Type);
Py_INCREF(DObject_Type.tp_base);
PyModule_AddObject(m, "DObject", (PyObject *)&DObject_Type);
+
+ return m;
}
diff --git a/lib/talloc/test_pytalloc.py b/lib/talloc/test_pytalloc.py
index 15863e0..a6cfaab 100644
--- a/lib/talloc/test_pytalloc.py
+++ b/lib/talloc/test_pytalloc.py
@@ -18,14 +18,14 @@ def dummy_func():
class TallocTests(unittest.TestCase):
def test_report_full(self):
# report_full is hardcoded to print to stdout, so use a subprocess
- output = subprocess.check_output([
+ output = str(subprocess.check_output([
sys.executable, '-c',
"""if True:
import talloc, _test_pytalloc
obj = _test_pytalloc.new()
talloc.report_full(obj)
"""
- ])
+ ]))
self.assertIn("full talloc report on 'talloc.Object", output)
self.assertIn("This is a test string", output)
@@ -74,7 +74,7 @@ class TallocComparisonTests(unittest.TestCase):
def test_compare_different_types(self):
# object comparison falls back to comparing types
- if talloc.Object < _test_pytalloc.DObject:
+ if sys.version_info >= (3, 0) or talloc.Object < _test_pytalloc.DObject:
obj1 = _test_pytalloc.new()
obj2 = _test_pytalloc.DObject(dummy_func)
else:
@@ -82,10 +82,16 @@ class TallocComparisonTests(unittest.TestCase):
obj1 = _test_pytalloc.DObject(dummy_func)
self.assertFalse(obj1 == obj2)
self.assertTrue(obj1 != obj2)
- self.assertTrue(obj1 <= obj2)
- self.assertTrue(obj1 < obj2)
- self.assertFalse(obj1 >= obj2)
- self.assertFalse(obj1 > obj2)
+ if sys.version_info >= (3, 0):
+ self.assertRaises(TypeError, lambda: obj1 <= obj2)
+ self.assertRaises(TypeError, lambda: obj1 < obj2)
+ self.assertRaises(TypeError, lambda: obj1 >= obj2)
+ self.assertRaises(TypeError, lambda: obj1 > obj2)
+ else:
+ self.assertTrue(obj1 <= obj2)
+ self.assertTrue(obj1 < obj2)
+ self.assertFalse(obj1 >= obj2)
+ self.assertFalse(obj1 > obj2)
class TallocUtilTests(unittest.TestCase):
--
2.1.0
From 7aae8aedeef5908527afb6701a6263957cb22755 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori at redhat.com>
Date: Wed, 19 Nov 2014 14:50:07 +0100
Subject: [PATCH 13/13] pytalloc: Build for Python 3/extrapython
This enables building, installing & testing two versions for pytalloc,
for two versions of Python, at the same time.
Signed-off-by: Petr Viktorin <pviktori at redhat.com>
---
lib/talloc/pytalloc-util.pc.in | 2 +-
lib/talloc/pytalloc_guide.txt | 8 +++++++
lib/talloc/wscript | 50 +++++++++++++++++++++++++++++-------------
3 files changed, 44 insertions(+), 16 deletions(-)
diff --git a/lib/talloc/pytalloc-util.pc.in b/lib/talloc/pytalloc-util.pc.in
index b7426bb..b87c94e 100644
--- a/lib/talloc/pytalloc-util.pc.in
+++ b/lib/talloc/pytalloc-util.pc.in
@@ -6,6 +6,6 @@ includedir=@includedir@
Name: pytalloc-util
Description: Utility functions for using talloc objects with Python
Version: @TALLOC_VERSION@
-Libs: @LIB_RPATH@ -L${libdir} -lpytalloc-util
+Libs: @LIB_RPATH@ -L${libdir} -lpytalloc-util at PYTHON_SO_ABI_FLAG@
Cflags: -I${includedir}
URL: http://talloc.samba.org/
diff --git a/lib/talloc/pytalloc_guide.txt b/lib/talloc/pytalloc_guide.txt
index 755a52b..80968e0 100644
--- a/lib/talloc/pytalloc_guide.txt
+++ b/lib/talloc/pytalloc_guide.txt
@@ -20,6 +20,14 @@ for objects that wrap talloc-maintained memory in C. It won't write your
bindings for you but it will make it easier to write C bindings that involve
talloc, and take away some of the boiler plate.
+Python 3
+--------
+
+pytalloc can be used with Python 3. Usage from Python extension remains
+the same, but for the C utilities, the library to link to is tagged with
+Python's PEP3149 ABI tag, for example "pytalloc.cpython34m".
+To make a build for Python 3, configure with PYTHON=/usr/bin/python3.
+.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
pytalloc_Object
diff --git a/lib/talloc/wscript b/lib/talloc/wscript
index 0f21839..eba2db7 100644
--- a/lib/talloc/wscript
+++ b/lib/talloc/wscript
@@ -111,21 +111,26 @@ def build(bld):
manpages='man/talloc.3')
if not bld.CONFIG_SET('USING_SYSTEM_PYTALLOC_UTIL') and not bld.env.disable_python:
- bld.SAMBA_LIBRARY('pytalloc-util',
+ name = bld.pyembed_libname('pytalloc-util')
+ options = dict(
source='pytalloc_util.c',
public_deps='talloc',
- pyfeature='pyembed',
vnum=VERSION,
hide_symbols=True,
abi_directory='ABI',
- abi_match='pytalloc_*',
private_library=private_library,
- public_headers='pytalloc.h',
- pc_files='pytalloc-util.pc'
+ abi_match='pytalloc_*',
)
+
+ name = bld.pyembed_libname('pytalloc-util')
+ bld.SAMBA_LIBRARY(name,
+ pyfeature='pyembed',
+ pc_files='pytalloc-util.pc',
+ public_headers='pytalloc.h',
+ **options)
bld.SAMBA_PYTHON('pytalloc',
'pytalloc.c',
- deps='talloc pytalloc-util',
+ deps='talloc ' + name,
enabled=True,
realname='talloc.so')
@@ -136,21 +141,36 @@ def build(bld):
realname='_test_pytalloc.so',
install=False)
+ if bld.env['EXTRA_PYTHON']:
+ name = bld.pyembed_libname('pytalloc-util', extrapython=True)
+ bld.SAMBA_LIBRARY(name,
+ pyfeature='extrapyembed',
+ pc_files=None,
+ **options)
+
+ bld.SAMBA_PYTHON('extra-pytalloc',
+ 'pytalloc.c',
+ deps='talloc ' + name,
+ enabled=True,
+ realname='talloc.so',
+ extra_python=True)
+
+ bld.SAMBA_PYTHON('extra-test_pytalloc',
+ 'test_pytalloc.c',
+ deps='pytalloc',
+ enabled=True,
+ realname='_test_pytalloc.so',
+ install=False,
+ extra_python=True)
+
+
def test(ctx):
'''run talloc testsuite'''
import Utils, samba_utils
- env = samba_utils.LOAD_ENVIRONMENT()
cmd = os.path.join(Utils.g_module.blddir, 'talloc_testsuite')
ret = samba_utils.RUN_COMMAND(cmd)
print("testsuite returned %d" % ret)
- if 'USING_SYSTEM_PYTALLOC_UTIL' not in env.defines and not env.disable_python:
- cmd = "PYTHONPATH=%s %s test_pytalloc.py" % (
- os.path.join(Utils.g_module.blddir, 'python'),
- env['PYTHON'],
- )
- pyret = samba_utils.RUN_COMMAND(cmd)
- else:
- pyret = 0
+ pyret = samba_utils.RUN_PYTHON_TESTS(['test_pytalloc.py'])
print("python testsuite returned %d" % pyret)
sys.exit(ret or pyret)
--
2.1.0
More information about the samba-technical
mailing list