A plan to get to py3 (was: Re: Pure python3 for the AD DC for Samba 4.10 (5.0))

[Andrew Bartlett]

On Tue, 2018-08-07 at 17:20 +0100, Noel Power via samba-technical
wrote:
> Hi Andrew
> 
> 
> On 07/08/18 04:15, Andrew Bartlett wrote:
> > 
> 
> yes, I have seen some of these flap in others CI results too, however
> normally with 1 or 2 retries I can get the build_samba job to complete
> successfully (with other branches), this has not been the case for the
> last couple of weeks with my py3 wip branches. This makes me really nervous.

I certainly understand that.  

> > 
> > > With that job now taking ~310 minutes the turn around time is really
> > > torturous, something odd is going on there, I am running out of ideas
> > > about getting to the bottom of it. Of course running any of the
> > > problematic tests in question manually pass no problem on my own machine.
> > 
> > Yes, it really is hard.  Splitting up the main 'samba' build would be
> > worthwhile, but also a distraction.  
> > 
> > Let me know if I can help at all.
> 
> If we could get to the point of CI passing with some degree of
> reliability that would be fantastic. But like I said it appears the
> current unreliability seems to be with my branch (and especially post
> rebase)

So, here is a plan:

Please make a waf that works with 

PYTHON=python3 ./configure --enable-developer && make -j

Then change selftest/selftesthelpers.py:planpythontestsuite() to only
run python3 tests if py3_compatible is set and we are built with
python3, and only python2 tests if built primarily with python2.

This will then mean you can avoid the hacks around demote, and a full
autobuild can run in both modes, just change the environment variable
in .gitlab-ci.yml

Then change script/autobuild.py to duplicate all the environments that
run python tests into a -py3 environment, eg 'samba-none-py3':

Add a extra 0 to the random-sleep lines to spread out the build load,
eg:  random-sleep.sh 60 6000

Bonus points for using a python variable to do that.

Add the string 'python3' to the TESTS= on those duplicated
environments, so they only run python tests, not smbtorture tests. 

    "samba-none-env-py3" : [
                      ("random-sleep", "script/random-sleep.sh 60 600",
"text/plain"),
                      ("configure", "PYTHON=python3
./configure.developer --with-selftest-prefix=./bin/ab" +
samba_configure_params, "text/plain"),
                      ("make", "make -j", "text/plain"),
                      ("test", "make test "
                       "FAIL_IMMEDIATELY=1 "
                       "TESTS='python3 --include-env=none'",
                       "text/plain")],

Also add to defaulttasks and builddirs. 

  Match with changes in the .gitlab-ci.yml file. 

build_samba_none_env_py3:
  <<: *shared_template
  script:
    # this one takes about 1 hours to finish
    - python3 script/autobuild.py samba-none-env-py3    --verbose --
nocleanup --keeplogs --tail --testbase /tmp/samba-testbase

This allows the python tests to run in parallel, so they don't increase
the complexity of any given run, just scale out horizontally.

Then make that pass.

Then make planpythontestsuite add .python2 only for python2 builds (ie,
invert the naming).   Fix up knownfail/flapping if required.  Then
change .gitlab-ci.yml and script/autobuild.py to make python2 the
'extra' environment with TESTS=python2.  

Then we have a Samba that builds and runs with python3 and we have
confidence that the python2 only build still works.  Then we can
discuss if we need to maintain that, after it becomes the secondary
target rather than while it is the 'primary'.

Then remove the EXTRA_PYTHON thing.  Metze indicated that he was
willing to accept that we only do only python at a time, and this is a
reasonable step.  I hope to understand why we need python 2.7 in the
long term, and get consensus to remove it, but this is useful anyway.

I know it is a long list, and you still need to reduce that patch
queue, but splitting up the build environments will make it easier to
get past CI, and reduce the internal waf complexity.

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba