Difficulties bringing waf15 updates into Samba (was: Re: [PATCH]?build scripts enhancements)

Michael Adam obnox at samba.org
Fri Jun 26 01:35:13 MDT 2015


On 2015-06-26 at 07:52 +0200, Thomas Nagy wrote:
> On Fri, 26 Jun 2015 12:28:05 +1200, Andrew Bartlett wrote:
> > On Fri, 2015-06-19 at 18:36 +0200, Thomas Nagy wrote:
> > > Please apply the patches attached to this message and update your Waf 1.5 copy from the source repository https://github.com/waf-project/waf.waf15
> > > 
> > > Thomas
> > 
> > If we update waf again, we currently have to keep the local patch to
> > python.py (here the reverse, removing it as part of the update):
> > 
> > diff --git a/third_party/waf/wafadmin/Tools/python.py b/third_party/waf/wafadmin/Tools/python.py
> > index cd96b65..b8cfcc2 100644
> > --- a/third_party/waf/wafadmin/Tools/python.py
> > +++ b/third_party/waf/wafadmin/Tools/python.py
> > @@ -195,19 +195,6 @@ LDVERSION = %r
> >         python_LIBDIR, python_LIBPL, INCLUDEPY, Py_ENABLE_SHARED, python_MACOSX_DEPLOYMENT_TARGET,
> >         python_LDVERSION))
> >  
> > -       # Allow some python overrides from env vars for cross-compiling
> > -       os_env = dict(os.environ)
> > -
> > -       override_python_LDFLAGS = os_env.get('python_LDFLAGS', None)
> > -       if override_python_LDFLAGS is not None:
> > -               conf.log.write("python_LDFLAGS override from environment = %r\n" % (override_python_LDFLAGS))
> > -               python_LDFLAGS = override_python_LDFLAGS
> > -
> > -       override_python_LIBDIR = os_env.get('python_LIBDIR', None)
> > -       if override_python_LIBDIR is not None:
> > -               conf.log.write("python_LIBDIR override from environment = %r\n" % (override_python_LIBDIR))
> > -               python_LIBDIR = override_python_LIBDIR
> > -
> >         if python_MACOSX_DEPLOYMENT_TARGET:
> >                 conf.env['MACOSX_DEPLOYMENT_TARGET'] = python_MACOSX_DEPLOYMENT_TARGET
> >                 conf.environ['MACOSX_DEPLOYMENT_TARGET'] = python_MACOSX_DEPLOYMENT_TARGET
> > 
> > Do you have any thoughts on a better way to do this?
> > 
> > We also can't import the whole waf tree, or carry it as a submodule, as
> > that would bring in waf/doc/book which is non-free.  (Carrying that
> > would cause all distributions, not just Debian, to then have to expunge
> > it from our tarball, and may even cause issues with our obligations to
> > our fiscal sponsor, which strictly requires that Samba distributes only
> > free software).
> 
> Are the new git submodules in Samba causing some confusion?

Afaik, we do not use git submodules (yet?) for third_party/ in Samba.

> There used to be an "update.py" script in the Samba tree to
> cherry-pick relevant changes from external trees.
>
> I thought that you would either use that script or simply
> re-apply the changes extracted by git format-patch.
> 
> https://github.com/waf-project/waf.waf15/commit/a2ca9d5d505b20211bac328c167f0d99353af63f.patch
> https://github.com/waf-project/waf.waf15/commit/2dfed29ed9d4d11cf291baba9384371e4e97c1c8.patch

The script is still there as third_party/update.sh .
But this does not cherry-pick but bluntly copies
contents from an external tree.

The relevant portion for waf is this:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
THIRD_PARTY_DIR="`dirname $0`"
#...
WORKDIR="`mktemp -d`"
#...
echo "Updating waf..."
git clone git://git.samba.org/third_party/waf.waf15/ "$WORKDIR/waf"
rm -rf "$WORKDIR/waf/.git"
rsync -C -avz --delete "$WORKDIR/waf/" "$THIRD_PARTY_DIR/waf/"

rm -rf "$WORKDIR"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The important bit is that we fetch from a clone on samba.org
(we do that for almost all our third_party trees) in order to
not blindly take patches from upstream but stick with a
fixed upstream state that we know is working until wil conciously
choose to take new upstream patches.

That means that taking new upstream waf involves a manual process
to update that clone on samba.org. You were part of the discussion
recently, when I asked Jelmer about it, because he owns these
clones. Apparently, Jelmer has already updated the samba.org
clone of waf15. But unfortunately, we can't take it verbatim yet,
since as Andrew pointed out, there is one patch missing
that we would otherwise lose:

https://git.samba.org/?p=samba.git;a=patch;h=f8c179b80e1a95ce4eec60108a915fb615efab39

(This is the patch in the old buildtools/wafadmin/ tree before the move.)

So we have three options:

1. We cherry-pick your patches on top of our third_party/waf tree.
2. We run the update script and re-apply the python patch on
   third_party/waf.
3. You take the above into waf15 upstream and then we can take the
   upstream as is, just running update.sh.

I would certainly prefer the third option.
Would you consinder taking that patch?

If not so, I'd probably try to go with option #2, since
you have already taken everyting else that we have on top.

Cheers - Michael

> 
> Thomas
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20150626/7ea6b7d9/attachment.pgp>


More information about the samba-technical mailing list