Samba and Python2
Tim Beale
timbeale at catalyst.net.nz
Fri Feb 15 01:48:11 UTC 2019
I've updated the WHATSNEW text with Metze's feedback. Updated blurb
attached. Patch uploaded to the bug:
https://bugzilla.samba.org/show_bug.cgi?id=13785
Metze, unfortunately the '-py2' magic was only used for running the
tests using python2, not building the code itself. samba-buildpy2-only
was the only target that was actually using PYTHON=python2 in the
make/configure commands.
At first I tried using the same '-py2' approach, i.e. adding a new
replaceable '${PY_VERSION}' to the nopython job. But I quickly realized
this made things quite ugly, and made it more likely I'd mess something
up. So I just ended up duplicating the existing nopython job instead.
Updated patch-set attached. New CI link:
https://gitlab.com/samba-team/devel/samba/pipelines/47606161
I think adding some basic tests for nopython is a good idea. However,
this doesn't seem trivial - all the make test framework is based around
subunit/smbtorture, neither of which we seem to build with
disable-python. I'll dig into this some more, but I think in the
meantime the new nopython-py2 target is no worse than the py3 nopython
target is already.
On 14/02/19 8:59 PM, Stefan Metzmacher wrote:
> Am 14.02.19 um 05:13 schrieb Tim Beale via samba-technical:
>> I've attached an updated patch-set to drop the py2 jobs from CI
>> (.gitlab-ci.yml just got substantially reworked in master).
>>
>> New CI link: https://gitlab.com/samba-team/devel/samba/pipelines/47451100
>>
>> On 13/02/19 4:31 PM, Tim Beale via samba-technical wrote:
>>> Hi,
>>>
>>> Just following up on this. How about something like the following? It
>>> tries to merge what was discussed with what was already in the WHATSNEW.
>>>
>>> python3 support
>>> ---------------
>>> This is the first release of Samba which has full support for Python 3.
>>> Samba 4.10 still has support for Python 2, however, Python 3 will be used by
>>> default, i.e. 'configure' & 'make' will execute using python3.
>>>
>>> To build Samba with python2 you *must* set the 'PYTHON' environment variable
>>> for both the 'configure' and 'make' steps, i.e.
>>> 'PYTHON=python2 ./configure'
>>> 'PYTHON=python2 make'
>>> This will override the python3 default.
>>>
>>> Alternatively, it is possible to produce Samba Python bindings for both
>>> Python 2 and Python 3. To do so, specify '--extra-python=/usr/bin/python2'
>>> as part of the 'configure' command. Note that python3 will still be used as
>>> the default in this case.
>>>
>>> Note that Samba 4.10 supports Python 3.4 onwards.
>>>
>>> Future Python support
>>> ---------------------
>>> Samba 4.10 will be the last release that comes with support for Python 2.
>>> Unfortunately, the Samba Team doesn't have the resources to support both
>>> Python 2 and Python 3 long-term.
>>>
>>> Samba 4.11 will not have any support for Python 2. This means if you use
>>> Python 2 bindings it is time to migrate to Python 3 now.
> I think we should make it clear that Python 2 and also any Python 3
> version would be enough to build --without-python.
>
>>> Also note that Samba 4.11 will only support Python 3.6 onwards.
> I'd say '... will most likely only support ...'
>
> We should actually try to keep 3.4+ (or at least 3.5+) as long as we
> can. E.g. Ubuntu 16.04 uses 3.5.2 and I'm pretty sure it will be
> security updates even if 3.5 is no longer supported upstream.
> Debian jessie only has 3.4.2 and stretch 3.5.3.
>
> If we really make use of new async/await in important code sections
> we can discuss it again.
>
> I think samba-nopython-py should not be different than samba-nopython.
> Can't we just keep the magic '-py2' handling for that?
>
> I think we should also add some basic tests to it, instead of just
> compiling it.
>
> metze
>
-------------- next part --------------
python3 support
---------------
This is the first release of Samba which has full support for Python 3.
Samba 4.10 still has support for Python 2, however, Python 3 will be used by
default, i.e. 'configure' & 'make' will execute using python3.
To build Samba with python2 you *must* set the 'PYTHON' environment variable
for both the 'configure' and 'make' steps, i.e.
'PYTHON=python2 ./configure'
'PYTHON=python2 make'
This will override the python3 default.
Alternatively, it is possible to produce Samba Python bindings for both
Python 2 and Python 3. To do so, specify '--extra-python=/usr/bin/python2'
as part of the 'configure' command. Note that python3 will still be used as
the default in this case.
Note that Samba 4.10 supports Python 3.4 onwards.
Future Python support
---------------------
Samba 4.10 will be the last release that comes with full support for
Python 2. Unfortunately, the Samba Team doesn't have the resources to support
both Python 2 and Python 3 long-term.
Samba 4.11 will not have any runtime support for Python 2. This means if
you use Python 2 bindings it is time to migrate to Python 3 now.
If you are building Samba using the '--disable-python' option (i.e. you're
excluding all the run-time Python support), then this will continue to work
on a system that supports either python2 or python3.
Also note that Samba 4.11 will most likely only support Python 3.6 onwards.
-------------- next part --------------
From f7e1876a04e7028806f6a78000f7b6621d983653 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Wed, 13 Feb 2019 12:12:18 +1300
Subject: [PATCH 1/7] autobuild: Drop py2 autobuild jobs
Samba v4.11 will no longer support python2, so let's drop the autobuild
jobs. This will save some gitlab/sn-devel time and money, as it's less
work for CI to do.
Note that this highlights some previous inconsistencies:
- samba-none-env-py2 was being built for gitlab but not sn-devel.
- samba-nt4-py2 was being built for sn-devel but not gitlab
I've left samba-buildpy2-only for now, which will be addressed in a
subsequent patch.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
.gitlab-ci-private.yml | 8 --------
.gitlab-ci.yml | 13 -------------
script/autobuild.py | 6 ------
3 files changed, 27 deletions(-)
diff --git a/.gitlab-ci-private.yml b/.gitlab-ci-private.yml
index 4a6e503..4e83c19 100644
--- a/.gitlab-ci-private.yml
+++ b/.gitlab-ci-private.yml
@@ -18,10 +18,6 @@ samba:
extends: .private_template
# this one takes about 4 hours to finish
-samba-py2:
- extends: .private_template
- # this one takes about 4 hours to finish
-
samba-fileserver:
extends: .private_template
# this one takes about 1 hours to finish
@@ -30,9 +26,5 @@ samba-ad-dc:
extends: .private_template
# this one takes about 1 hours to finish
-samba-ad-dc-py2:
- extends: .private_template
- # this one takes about 1 hours to finish
-
samba-nt4:
extends: .private_template
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 8dce4af..c11c9b5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -61,10 +61,6 @@ samba-none-env:
extends: .shared_template
# this one takes about 1 hours to finish
-samba-none-env-py2:
- extends: .shared_template
- # this one takes about 1 hours to finish
-
samba-nopython:
extends: .shared_template
@@ -84,16 +80,9 @@ samba-ad-dc-2:
samba-ad-dc-backup:
extends: .shared_template
-samba-ad-dc-2-py2:
- extends: .shared_template
- # this one takes about 1 hours to finish
-
samba-libs:
extends: .shared_template
-samba-libs-py2:
- extends: .shared_template
-
samba-static:
extends: .shared_template
@@ -110,5 +99,3 @@ samba-ad-dc-ntvfs:
extends: .shared_template
# this one takes about 100 mins to finish
-samba-ad-dc-ntvfs-py2:
- extends: .shared_template
diff --git a/script/autobuild.py b/script/autobuild.py
index b43adcf..30bc643 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -36,23 +36,17 @@ cleanup_list = []
builddirs = {
"ctdb": "ctdb",
"samba": ".",
- "samba-py2": ".",
"samba-nt4": ".",
- "samba-nt4-py2": ".",
"samba-fileserver": ".",
"samba-xc": ".",
"samba-o3": ".",
"samba-ctdb": ".",
"samba-libs": ".",
- "samba-libs-py2": ".",
"samba-static": ".",
"samba-none-env": ".",
"samba-ad-dc": ".",
- "samba-ad-dc-py2": ".",
"samba-ad-dc-ntvfs": ".",
- "samba-ad-dc-ntvfs-py2": ".",
"samba-ad-dc-2": ".",
- "samba-ad-dc-2-py2": ".",
"samba-ad-dc-backup": ".",
"samba-systemkrb5": ".",
"samba-nopython": ".",
--
2.7.4
From eeafacb042cb87db076f3d26a8fee72baf6a27b5 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Wed, 13 Feb 2019 12:41:34 +1300
Subject: [PATCH 2/7] autobuild: Update variable name to make more sense
When we switched from python2 being the default to python3, we didn't
update this variable name. It's now handling the python2 case, but it's
a boolean flag named 'py3', which is rather confusing.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
script/autobuild.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/script/autobuild.py b/script/autobuild.py
index 30bc643..72050e0 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -457,9 +457,9 @@ def run_cmd(cmd, dir=".", show=None, output=False, checkfail=True):
class builder(object):
'''handle build of one directory'''
- def __init__(self, name, sequence, cp=True, py3=False):
+ def __init__(self, name, sequence, cp=True, py2=False):
self.name = name
- self.py3 = py3
+ self.py2 = py2
if name in builddirs:
self.dir = builddirs[name]
else:
@@ -498,7 +498,7 @@ class builder(object):
(self.stage, self.cmd, self.output_mime_type) = self.sequence[self.next]
self.cmd = self.cmd.replace("${PYTHON_PREFIX}", get_python_lib(plat_specific=1, standard_lib=0, prefix=self.prefix))
self.cmd = self.cmd.replace("${PREFIX}", "--prefix=%s" % self.prefix)
- if self.py3:
+ if self.py2:
self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "%s" % extra_python)
# The trailing space is important
self.cmd = self.cmd.replace("${PY3_ONLY}", "python2 ")
@@ -542,7 +542,7 @@ class buildlist(object):
b = builder(n,
tasks[n[:-4]],
cp=n is not "pidl",
- py3=True)
+ py2=True)
else:
b = builder(n, tasks[n], cp=n is not "pidl")
self.tlist.append(b)
--
2.7.4
From 48649608c488e18ef60b0e0db50d80c63157619c Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 15 Feb 2019 11:44:21 +1300
Subject: [PATCH 3/7] autobuild: Remove the PY3_ONLY variable
This variable is no longer needed as all the tests run using python3
now.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
script/autobuild.py | 23 ++++++++++-------------
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/script/autobuild.py b/script/autobuild.py
index 72050e0..5df8043 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -95,7 +95,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--exclude-env=none "
"--exclude-env=nt4_dc "
"--exclude-env=nt4_member "
@@ -126,7 +126,7 @@ tasks = {
("configure", "./configure.developer --without-ads --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=nt4_dc --include-env=nt4_member'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
@@ -136,7 +136,7 @@ tasks = {
("configure", "./configure.developer --without-ad-dc --without-ldap --without-ads --without-json --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=fileserver'", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain")],
@@ -144,7 +144,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=ad_dc "
"--include-env=fl2003dc "
"--include-env=fl2008r2dc "
@@ -157,7 +157,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=chgdcpass "
"--include-env=vampire_2000_dc "
"--include-env=fl2000dc "
@@ -173,7 +173,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=ad_dc_ntvfs "
"'",
"text/plain"),
@@ -185,7 +185,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=backupfromdc "
"--include-env=restoredc "
"--include-env=renamedc "
@@ -216,7 +216,7 @@ tasks = {
("configure", "ADDITIONAL_CFLAGS='-O3 -Wp,-D_FORTIFY_SOURCE=2' ./configure.developer --with-selftest-prefix=./bin/ab --abi-check-disable" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make quicktest FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=ad_dc'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
@@ -277,7 +277,7 @@ tasks = {
("make", "make -j", "text/plain"),
("test", "make test "
"FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=none'",
"text/plain")],
@@ -308,7 +308,7 @@ tasks = {
# we currently cannot run a full make test, a limited list of tests could be run
# via "make test TESTS=sometests"
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='${PY3_ONLY}"
+ "TESTS='"
"--include-env=ktest'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
@@ -500,11 +500,8 @@ class builder(object):
self.cmd = self.cmd.replace("${PREFIX}", "--prefix=%s" % self.prefix)
if self.py2:
self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "%s" % extra_python)
- # The trailing space is important
- self.cmd = self.cmd.replace("${PY3_ONLY}", "python2 ")
else:
self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "")
- self.cmd = self.cmd.replace("${PY3_ONLY}", "")
self.cmd = self.cmd.replace("${PREFIX_DIR}", "%s" % self.prefix)
self.cmd = self.cmd.replace("${TESTS}", options.restrict_tests)
# if self.output_mime_type == "text/x-subunit":
--
2.7.4
From 76197a42f65e392e8dfb2dabbbcf22cbf4bf19e3 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 15 Feb 2019 11:45:53 +1300
Subject: [PATCH 4/7] autobuild: Tidy up unnecessary line-breaks in 'TESTS='
Now that we've dropped the {PY3_ONLY} variable, there's no need for
line-breaks in some of the 'TESTS=' values. We can tidy this up a bit.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
script/autobuild.py | 31 ++++++++++---------------------
1 file changed, 10 insertions(+), 21 deletions(-)
diff --git a/script/autobuild.py b/script/autobuild.py
index 5df8043..efaf884 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -95,8 +95,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--exclude-env=none "
+ "TESTS='--exclude-env=none "
"--exclude-env=nt4_dc "
"--exclude-env=nt4_member "
"--exclude-env=ad_dc "
@@ -126,8 +125,7 @@ tasks = {
("configure", "./configure.developer --without-ads --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=nt4_dc --include-env=nt4_member'", "text/plain"),
+ "TESTS='--include-env=nt4_dc --include-env=nt4_member'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
("clean", "make clean", "text/plain")],
@@ -136,16 +134,14 @@ tasks = {
("configure", "./configure.developer --without-ad-dc --without-ldap --without-ads --without-json --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=fileserver'", "text/plain"),
+ "TESTS='--include-env=fileserver'", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain")],
"samba-ad-dc": [("random-sleep", "script/random-sleep.sh 60 600", "text/plain"),
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=ad_dc "
+ "TESTS='--include-env=ad_dc "
"--include-env=fl2003dc "
"--include-env=fl2008r2dc "
"--include-env=ad_member "
@@ -157,8 +153,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=chgdcpass "
+ "TESTS='--include-env=chgdcpass "
"--include-env=vampire_2000_dc "
"--include-env=fl2000dc "
"--include-env=ad_dc_no_nss "
@@ -173,9 +168,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=ad_dc_ntvfs "
- "'",
+ "TESTS='--include-env=ad_dc_ntvfs'",
"text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain")],
@@ -185,8 +178,7 @@ tasks = {
("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=backupfromdc "
+ "TESTS='--include-env=backupfromdc "
"--include-env=restoredc "
"--include-env=renamedc "
"--include-env=offlinebackupdc "
@@ -216,8 +208,7 @@ tasks = {
("configure", "ADDITIONAL_CFLAGS='-O3 -Wp,-D_FORTIFY_SOURCE=2' ./configure.developer --with-selftest-prefix=./bin/ab --abi-check-disable" + samba_configure_params, "text/plain"),
("make", "make -j", "text/plain"),
("test", "make quicktest FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=ad_dc'", "text/plain"),
+ "TESTS='--include-env=ad_dc'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
("clean", "make clean", "text/plain")],
@@ -277,8 +268,7 @@ tasks = {
("make", "make -j", "text/plain"),
("test", "make test "
"FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=none'",
+ "TESTS='--include-env=none'",
"text/plain")],
"samba-static": [
@@ -308,8 +298,7 @@ tasks = {
# we currently cannot run a full make test, a limited list of tests could be run
# via "make test TESTS=sometests"
("test", "make test FAIL_IMMEDIATELY=1 "
- "TESTS='"
- "--include-env=ktest'", "text/plain"),
+ "TESTS='--include-env=ktest'", "text/plain"),
("install", "make install", "text/plain"),
("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
("clean", "make clean", "text/plain")
--
2.7.4
From fea9360d7a871cc4feb9fed12355895b4d7e1347 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 15 Feb 2019 11:58:51 +1300
Subject: [PATCH 5/7] autobuild: Remove ${EXTRA_PYTHON} variable
We no longer build the python2 bindings, only python3. So we can get rid
of this variable now.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
script/autobuild.py | 25 ++++++++-----------------
1 file changed, 8 insertions(+), 17 deletions(-)
diff --git a/script/autobuild.py b/script/autobuild.py
index efaf884..2d37cff 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -65,20 +65,15 @@ if os.environ.get("AUTOBUILD_SKIP_SAMBA_O3", "0") == "1":
defaulttasks.remove("samba-o3")
ctdb_configure_params = " --enable-developer --picky-developer ${PREFIX}"
-samba_configure_params = " --picky-developer ${PREFIX} ${EXTRA_PYTHON} --with-profiling-data"
+samba_configure_params = " --picky-developer ${PREFIX} --with-profiling-data"
samba_libs_envvars = "PYTHONPATH=${PYTHON_PREFIX}:$PYTHONPATH"
samba_libs_envvars += " PKG_CONFIG_PATH=$PKG_CONFIG_PATH:${PREFIX_DIR}/lib/pkgconfig"
samba_libs_envvars += " ADDITIONAL_CFLAGS='-Wmissing-prototypes'"
samba_libs_configure_base = samba_libs_envvars + " ./configure --abi-check --enable-debug --picky-developer -C ${PREFIX}"
-samba_libs_configure_libs = samba_libs_configure_base + " --bundled-libraries=cmocka,popt,NONE ${EXTRA_PYTHON}"
+samba_libs_configure_libs = samba_libs_configure_base + " --bundled-libraries=cmocka,popt,NONE"
samba_libs_configure_bundled_libs = " --bundled-libraries=!talloc,!pytalloc-util,!tdb,!pytdb,!ldb,!pyldb,!pyldb-util,!tevent,!pytevent,!popt"
-samba_libs_configure_samba = samba_libs_configure_base + samba_libs_configure_bundled_libs + " ${EXTRA_PYTHON}"
-
-if os.environ.get("AUTOBUILD_NO_EXTRA_PYTHON", "0") == "1":
- extra_python = ""
-else:
- extra_python = "--extra-python=/usr/bin/python2"
+samba_libs_configure_samba = samba_libs_configure_base + samba_libs_configure_bundled_libs
tasks = {
"ctdb": [("random-sleep", "../script/random-sleep.sh 60 600", "text/plain"),
@@ -346,11 +341,11 @@ tasks = {
"ldb": [
("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"),
- ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"),
+ ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"),
("make", "make", "text/plain"),
("install", "make install", "text/plain"),
("test", "make test", "text/plain"),
- ("configure-no-lmdb", "./configure --enable-developer --without-ldb-lmdb -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"),
+ ("configure-no-lmdb", "./configure --enable-developer --without-ldb-lmdb -C ${PREFIX}", "text/plain"),
("make-no-lmdb", "make", "text/plain"),
("install-no-lmdb", "make install", "text/plain"),
("check-clean-tree", "../../script/clean-source-tree.sh", "text/plain"),
@@ -359,7 +354,7 @@ tasks = {
"tdb": [
("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"),
- ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"),
+ ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"),
("make", "make", "text/plain"),
("install", "make install", "text/plain"),
("test", "make test", "text/plain"),
@@ -369,7 +364,7 @@ tasks = {
"talloc": [
("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"),
- ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"),
+ ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"),
("make", "make", "text/plain"),
("install", "make install", "text/plain"),
("test", "make test", "text/plain"),
@@ -389,7 +384,7 @@ tasks = {
"tevent": [
("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"),
- ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"),
+ ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"),
("make", "make", "text/plain"),
("install", "make install", "text/plain"),
("test", "make test", "text/plain"),
@@ -487,10 +482,6 @@ class builder(object):
(self.stage, self.cmd, self.output_mime_type) = self.sequence[self.next]
self.cmd = self.cmd.replace("${PYTHON_PREFIX}", get_python_lib(plat_specific=1, standard_lib=0, prefix=self.prefix))
self.cmd = self.cmd.replace("${PREFIX}", "--prefix=%s" % self.prefix)
- if self.py2:
- self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "%s" % extra_python)
- else:
- self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "")
self.cmd = self.cmd.replace("${PREFIX_DIR}", "%s" % self.prefix)
self.cmd = self.cmd.replace("${TESTS}", options.restrict_tests)
# if self.output_mime_type == "text/x-subunit":
--
2.7.4
From d19e2fcbb22f1c55b687c62fc87e6b60fe6b9722 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 15 Feb 2019 12:17:49 +1300
Subject: [PATCH 6/7] autobuild: Replace samba-buildpy2-only with
samba-nopython-py2
For Samba 4.11, the minimum python2 functionality we will support (for
now, at least - we may change our minds) is for the --disable-python
target, i.e. if you're excluding all the python functionality from
samba, then WAF should still support being built with python2.
The use case here is old unix platforms that want to use smbd, but don't
have python3 support.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
.gitlab-ci.yml | 6 +++---
script/autobuild.py | 40 ++++++++++++++++++++++++++++++++--------
2 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c11c9b5..d999cf4 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -64,6 +64,9 @@ samba-none-env:
samba-nopython:
extends: .shared_template
+samba-nopython-py2:
+ extends: .shared_template
+
samba-systemkrb5:
extends: .shared_template
@@ -92,9 +95,6 @@ ctdb:
samba-ctdb:
extends: .shared_template
-samba-buildpy2-only:
- extends: .shared_template
-
samba-ad-dc-ntvfs:
extends: .shared_template
# this one takes about 100 mins to finish
diff --git a/script/autobuild.py b/script/autobuild.py
index 2d37cff..91098f5 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -50,7 +50,7 @@ builddirs = {
"samba-ad-dc-backup": ".",
"samba-systemkrb5": ".",
"samba-nopython": ".",
- "samba-buildpy2-only": ".",
+ "samba-nopython-py2": ".",
"ldb": "lib/ldb",
"tdb": "lib/tdb",
"talloc": "lib/talloc",
@@ -337,7 +337,38 @@ tasks = {
("libs-clean", "make clean", "text/plain")
],
+ # check we can do the same thing using python2
+ "samba-nopython-py2": [
+ ("random-sleep", "script/random-sleep.sh 60 600", "text/plain"),
+ ("configure", "PYTHON=python2 ./configure.developer --picky-developer ${PREFIX} --with-profiling-data --disable-python --without-ad-dc", "text/plain"),
+ ("make", "PYTHON=python2 make -j", "text/plain"),
+ ("install", "PYTHON=python2 make install", "text/plain"),
+ ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
+ ("clean", "PYTHON=python2 make clean", "text/plain"),
+
+ ("talloc-configure", "cd lib/talloc && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"),
+ ("talloc-make", "cd lib/talloc && PYTHON=python2 make", "text/plain"),
+ ("talloc-install", "cd lib/talloc && PYTHON=python2 make install", "text/plain"),
+ ("tdb-configure", "cd lib/tdb && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"),
+ ("tdb-make", "cd lib/tdb && PYTHON=python2 make", "text/plain"),
+ ("tdb-install", "cd lib/tdb && PYTHON=python2 make install", "text/plain"),
+
+ ("tevent-configure", "cd lib/tevent && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"),
+ ("tevent-make", "cd lib/tevent && PYTHON=python2 make", "text/plain"),
+ ("tevent-install", "cd lib/tevent && PYTHON=python2 make install", "text/plain"),
+
+ ("ldb-configure", "cd lib/ldb && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"),
+ ("ldb-make", "cd lib/ldb && PYTHON=python2 make", "text/plain"),
+ ("ldb-install", "cd lib/ldb && PYTHON=python2 make install", "text/plain"),
+
+ # retry against installed library packages
+ ("libs-configure", "PYTHON=python2 " + samba_libs_configure_base + samba_libs_configure_bundled_libs + " --disable-python --without-ad-dc", "text/plain"),
+ ("libs-make", "PYTHON=python2 make -j", "text/plain"),
+ ("libs-install", "PYTHON=python2 make install", "text/plain"),
+ ("libs-check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
+ ("libs-clean", "PYTHON=python2 make clean", "text/plain")
+ ],
"ldb": [
("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"),
@@ -403,13 +434,6 @@ tasks = {
("check-clean-tree", "../script/clean-source-tree.sh", "text/plain"),
("clean", "make clean", "text/plain")],
- "samba-buildpy2-only": [("random-sleep", "script/random-sleep.sh 60 600", "text/plain"),
- ("configure", "PYTHON='python' ./configure.developer --with-selftest-prefix=./bin/ab " + samba_configure_params, "text/plain"),
- ("make", "PYTHON='python' make -j", "text/plain"),
- ("install", "PYTHON='python' make install", "text/plain"),
- ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"),
- ("clean", "PYTHON='python' make clean", "text/plain")],
-
# these are useful for debugging autobuild
'pass': [("pass", 'echo passing && /bin/true', "text/plain")],
--
2.7.4
From 668bced9c24428062267622672bfbd25a04b25a8 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 15 Feb 2019 12:20:10 +1300
Subject: [PATCH 7/7] autobuild: Drop 'py2' flag
This isn't used any more. It was only being set, never referenced.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
script/autobuild.py | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)
diff --git a/script/autobuild.py b/script/autobuild.py
index 91098f5..f2a2949 100755
--- a/script/autobuild.py
+++ b/script/autobuild.py
@@ -465,9 +465,8 @@ def run_cmd(cmd, dir=".", show=None, output=False, checkfail=True):
class builder(object):
'''handle build of one directory'''
- def __init__(self, name, sequence, cp=True, py2=False):
+ def __init__(self, name, sequence, cp=True):
self.name = name
- self.py2 = py2
if name in builddirs:
self.dir = builddirs[name]
else:
@@ -539,13 +538,7 @@ class buildlist(object):
os.environ['AUTOBUILD_RANDOM_SLEEP_OVERRIDE'] = '1'
for n in tasknames:
- if n not in tasks and n.endswith("-py2"):
- b = builder(n,
- tasks[n[:-4]],
- cp=n is not "pidl",
- py2=True)
- else:
- b = builder(n, tasks[n], cp=n is not "pidl")
+ b = builder(n, tasks[n], cp=n is not "pidl")
self.tlist.append(b)
if options.retry:
rebase_remote = "rebaseon"
--
2.7.4
More information about the samba-technical
mailing list