status of the s4 waf build - go to stage 3?

[tridge at samba.org]

Hi All,

Most developers building s4 now seem to have switched to the waf
build and everyone seems to be pretty happy with it.

At this stage it may make sense to go to "stage 3" for use of waf in
Samba4 (as per the original proposal). That would mean:

 1) that the waf build would be the recommended build

 2) that autogen.sh be renamed to autogen-autotools.sh and autogen.sh
 would setup the waf build

 3) that developers changing anything that affects Samba4 would be
 responsible for updating the wscript files if needed, instead of the
 config.mk files

There is a problem however. The merged build is not done yet, so if we
switch Samba4 to use waf, the merged build may bitrot. 

Right now I'm in two minds over this. The old system for Samba4 is
already bitrotting, and stopping the rot is very hard. It is failing
at least two tests on all platforms, and I think (but haven't yet
proved) that this may be caused by the "doubly instantiated global
variable" problem that is pretty deeply ingrained in the old build
system. If that is in fact the cause of the failures then it will be
very hard to fix, and I think it would be a waste of effort if we plan
on dropping the old Samba4 build system in the near future.

At the same time, I don't want to lose the merged build. Once the waf
s3 build is done I'd like to see the merged build use waf (assuming
we reach agreement on that, which isn't at all clear!). 

I'm also thinking about the next alpha of Samba4. Even if the two test
cases that are broken with the old build are not caused by the doubly
instantiated globals problem, we do know that doubly instantiated
variables do cause many very subtle bugs. I'd like those bugs to be
gone in the next Samba4 alpha, so I'd like the next alpha to use the
waf build. 

I think the only real solution is to modify the "stages" plan a
little. The original plan was that when we shifted to stage 3 that the
old build system in Samba4 would be only maintained by those
enthusiastic enough to maintain it. I didn't imagine that I'd be one
of those people :-)

I now think we should consider moving to stage 3 for Samba4/waf, but
we also commit to keeping the old build at least building until the
time when we are ready to switch the merged build over. We can do this
via the build farm, keeping both the old and new build systems in the
build farm, and then fixing build failures with the old build system
as they come up.

Note that the old build system for Samba4 currently supplies the
smbtorture for testing Samba3 in the build farm. We could either leave
the old build system supplying this, or we could switch the scripts to
using the smbtorture from the waf build. That would need some careful
testing to make sure it works as smbtorture in the waf build uses
shared libs.

Alternatively we could add a --nonshared-binary=smbtorture option,
which would tell the waf build system to not use shared libs for
linking smbtorture. That may be easier, and being able to ask for
specific binaries to be built non-shared could be useful anyway.

Comments?

Cheers, Tridge