WAF 2.x upgrade for 4.9

[Andrew Bartlett]

On Fri, 2018-07-06 at 09:13 +0300, Alexander Bokovoy wrote:
> On pe, 06 heinä 2018, Andrew Bartlett wrote:
> > On Fri, 2018-07-06 at 07:40 +0300, Alexander Bokovoy via samba-
> > technical wrote:
> > > On pe, 06 heinä 2018, Gary Lockyer via samba-technical wrote:
> > > > Can I ask why the urgency to get the WAF upgrade into 4.9.  It's going
> > > > to b much safer if this gets landed after the 4.9 cut-off.  My concern
> > > > is that changing the build system this close to a release is going to
> > > > cause all sorts of unnecessary pain.  Whereas if we wait until after the
> > > > 4.9 release we'll have 6 months to sort out any issues.
> > > 
> > > Python 2 will be dropped by Python upstream in 2020. Postponing a
> > > release to 2019 means we couldn't sort out distribution-wide Python 3
> > > migration for Samba and Samba AD targets until first half of 2019. This
> > > puts Fedora Project (in my case) into a dangerous state.
> > > 
> > > Waf upgrade is a first step in cleaning up Python 2 use and replacing it
> > > with Python 3-compatible code. Unfortunately, a reality is that it takes
> > > more than just porting Samba to py3 to do that. We need to make sure
> > > distribution as a whole deals well with that -- for example, while
> > > FreeIPA is using Python 3 already in Fedora 28 (current release) with
> > > select Samba Python modules and there we can be sure things work, the
> > > same couldn't be said for Samba AD path. Right now I couldn't even build
> > > Samba AD with Python 3 to test whether things would work at all.
> > 
> > I don't see how the distribution-wide stuff can come before the Python3
> > porting of the code is done.  Given that, why is the build-system step
> > urgent?  
> 
> It is extremely hard to maintain large patchsets which touch every
> aspect of Samba build off-tree.

Sure, which is why I urged this be merged soon.  There are still a few
things to sort out (not many thankfully).  Could it be done on Friday,
just after the freeze and branch?

> > 
> > > Note I'm talking about use outside of git master as we need to have
> > > supporting libraries packaged with upgraded waf. We have ~2 months until
> > > 4.9 release, I think it is sufficient time to sort out waf issues in
> > > prereleases given the analysis Andrew did to the resulting generated
> > > config files.
> > 
> > The concern is more that in the next 6 days, many developers will be
> > trying to land patches into master for the 4.9 freeze.  
> > 
> > So we don't have 2 months, we have 6 busy days.  I've seen how grumpy
> > folks get when their autobuilds stop working, and I wouldn't advise
> > being on the pointy end of that wrath. 
> 
> I don't mind taking responsibility for bugs where it is due.

I've been at the pointy end of 'you broke the autobuild', I wouldn't
recommend it. 

> Are you saying I'm now blocked from even merging this to master?

I think a merge after Samba 4.9rc1 on Friday would be the best
timeframe. 

> > This isn't the absolute last step before a distribution package builds
> > and operates with python3 is it?
> 
> Finding integration bugs is much harder when there is no base for
> integration at all.

Sure, but I'm still lost as to how this fits with a 4.9 release.  Can
you explain that bit some more?  It seems I'm missing something here,
because otherwise you wouldn't be at such urgency to have it in 4.9. 
Is there some other external requirement?

You say you want to get the dependent libs sorted.  Could we just make
extra releases of those after 4.9 freezes?

Can tarballs and packages be made for testing of 4.10pre1 and testing
done of that?  Given the distribution build can't be done with python3
(because python2 packages need to be produced, and the python3 packages
don't work), how does the new waf version change things?

Again, I'm just trying to understand, because this feels like more than
just the usual passion of 'please let this land, I can't stand it any
more'.  (I know that very well).  

Could you elaborate much further on the exact use case? 

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba