WAF 2.x upgrade for 4.9

[Noel Power]

Hi Alexander,

I got a chance to play with your branch, mostly I wanted to get to the
point of just trying to get *any* test to run with a pure python3 build
(where python3 is running waf and libpython3.x is the detected python
version). Unfortunately not at that point yet

You have understandably concentrated on getting waf upgraded so it
wasn't a surprise to find that trying to build with python3 failed. I am
attaching here my current set of patches to allow build and configure to
work with python3 (with some limitations... see below)

So what works, configure & build with python3 executing waf work.
However, I have failed to get a "FULL" python3 build to work, by 'FULL'
I mean where configure detects the version for libpython to build
against as 3.x. In other words I am building on a system with python2 &
python3 installed with the following cmd line examples

1)

   PYTHON=/usr/bin/python python ./buildtools/bin/waf configure
--enable-developer --picky-developer [WORKS]

2)

  PYTHON=/usr/bin/python python ./buildtools/bin/waf [WORKS]

3)

  PYTHON=/usr/bin/python3 python3 ./buildtools/bin/waf configure
--enable-developer --picky-developer [WORKS]

4)

  PYTHON=/usr/bin/python3 python3 ./buildtools/bin/waf [FAILS]

fails with

"ERROR: grouping library target
pytalloc-util.cpython-36m-x86_64-linux-gnu not declared in
samba_python.cpython-36m-x86_64-linux-gnu"

I haven't tried to get to the bottom of this yet, waf scares me :0

But I think this is a configuration (as in wbuild) problem because a
python3 build (using detected python lib version 2.7 works fine)

5)

  python3 ./buildtools/bin/waf configure --enable-developer
--picky-developer [WORKS]

6)

  python3 ./buildtools/bin/waf [WORKS]


3 & 4 pickup libpython3.x and build against that (and fails as described
above) My gut feeling this is actually a small (but possibly difficult
to find problem)

5 & 6 work, indicating that in the main running/compiling under python3
is working well

Perhaps you might consider adding the fixes attached to your branch ?


Hopefully (currently in progress) CI results will be good

https://gitlab.com/samba-team/devel/samba/pipelines/27063578


Noel



On 25/07/18 07:42, Alexander Bokovoy via samba-technical wrote:
> Hi Andrew,
>
> On ke, 25 heinä 2018, Andrew Bartlett wrote:
>> On Fri, 2018-07-06 at 22:31 +1200, Andrew Bartlett via samba-technical
>> wrote:
>>> On Fri, 2018-07-06 at 13:16 +0300, Alexander Bokovoy wrote:
>>>> On pe, 06 heinä 2018, Andrew Bartlett wrote:
>>>>> It isn't that simple.  For example make test strictly assumes that
>>>>> py3_compatible means run with extra_python.  In talloc and tdb both
>>>>> only look for PYTHON_SO_ABI_FLAG in the extra_python case.
>>>>>
>>>>> While these are 'only' bugs, it works against your case that something
>>>>> truly special happens if these patches are merged.
>>>> Can you give me more of those details you know about? We have no
>>>> recording of those anywhere, I think.
>>> No more details at this hour, that much just came up in a short code
>>> inspection looking for EXTRA_PYTHON and what it controlled. 
>> G'Day Alexander,
>>
>> Hopefully we can catch all these and then remove EXTRA_PYTHON when we
>> go py3 only.  Do you think we can get away with not building python2 at
>> all for talloc/tdb/ldb?
> I think we can. For example, python2-ldb is only required by ldb and
> samba in Rawhide. Same for python2-tevent, python2-tdb, and
> python2-talloc -- they all interconnected but there are no requirements
> outside of talloc/tdb/ldb/samba for python2 bindings in Rawhide.
>
>>>>> What I'm saying is you have sadly left this run just a bit too late. 
>>>>> There are still loose ends and there isn't the compelling outcome that
>>>>> would justify the disruption in the most stressful few days of the team
>>>>> calendar. 
>>>>>
>>>>> We all try and get just one more thing in, and now you know why I get
>>>>> grumpy when major features get held up 'with plenty of time before the
>>>>> release', because we all see how that time slips away.  But the last
>>>>> six days are crazy, I really don't think this should land this during
>>>>> them.
>>>> You are correct in that it might be a problem. Perhaps, I instead should
>>>> take a vacation and forget about doing this work until the end of 4.9
>>>> release time frame.
>>>>
>>>> Sorry for my frustration.
>>> It's OK.  We are all feeling the stress!  I'm required to be at work
>>> until Thursday (to review and guide the last things at our end,
>>> thankfully all much less dramatic) and I'm already counting down the
>>> days.
>>>
>>> Thanks you so much for all your hard work and I look forward to helping
>>> you land this in a couple of weeks after a good rest for us both.
>> I'm just wondering how this is going?  I'm really keen to help move
>> this into master now we have the release stream in a different branch.
> I've been busy with FreeIPA 4.7.0 release last few weeks and attempts to
> integrate it to Fedora this week, so not much progress on Samba side
> aside ensuring we aren't conflicting (I rebase daily). There is a
> blocker https://bugzilla.redhat.com/show_bug.cgi?id=1607635 that took a
> lot of my time this week (and is a hard nut, unfortunately).
>
> I'm going to look into a merge the rest of the week.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: waf2.0-py3-fixes.patch
Type: text/x-patch
Size: 28656 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180802/b8dc9a74/waf2.0-py3-fixes.bin>