proposal: merge waf build of s4 to master
mat at matws.net
Fri Apr 2 01:42:51 MDT 2010
On 02/04/2010 09:13, tridge at samba.org wrote:
> Hi All,
> A couple of weeks ago I proposed that any new build system for Samba
> be adopted in stages:
> I'd now like to propose that the waf build of Samba4 move to stage 2,
> which means merging it into master.
> As is explained in the staging proposal, at stage 2 the existing build
> system would still be the primary build system. Developers who are not
> interested in using the waf based build would have no responsibility
> to update the waf build scripts. I will keep the waf scripts up to
> date, along with anyone else who is interested in helping out (I have
> received patches from several people for the waf-wip branch, so it
> seems there is interest).
> The merge would change no files that affect the current build of
> Samba3 or Samba4. Only additional files are added (primarily the
> wscript files), and those are what allow the waf build to work.
> Once merged, developers would choose which build system to use by
> either using ./autogen.sh for the old build system, or
> ./autogen-waf.sh for the new system.
> Once the build system is chosen, then builds proceed as normal, with
> "./configure.developer&& make" working fine with both build systems.
> The build farm would continue to build with both build systems, as it
> does now. Once the s3 waf build that Kai is working on has progressed
> to the point where it is useful, I expect we will add a s3-waf build
> in the build farm as well.
> The reason I am proposing to merge into master now is that the waf
> based build has gotten to the point where it is very usable for
> general Samba4 development and usage. In fact, I've found using the
> old build system to be quite cumbersome by comparison, and I really
> miss the accurate dependencies, fast builds and small binaries when I
> do work in master.
> It also has become clear that the primary problems remaining with the
> waf based build are in fact bugs in the old build system, so it seems
> most appropriate to try to fix these while working in master.
> The sorts of bugs I am talking about are:
> - the hiding of debug messages. We in fact have quite a few things
> failing in "make test" that are never shown. This is because with
> the old build system debug.c gets linked into programs multiple
> times (especially via python modules). This results in DEBUG(1)
> messages not being shown, as the DEBUGLEVEL global gets
> instantiated multiple times in the code but only set once. That
> means some error messages just don't get shown even though they
> - duplicated talloc contexts. Andrew and I have found bugs due to
> duplicate globals such as the iconv convenience globals. We have
> fixed some of these in master, but more remain. The python.winreg
> failure in the build farm for the waf-wip branch seems to be one of
> these sorts of bugs, and I am still trying to chase it down.
> You can see these types of bugs by this message in the waf-wip build
> when running make test:
> ndr_pull_error(11): ndr_pull_relative_ptr1 rel_offset(758459697)> ndr->data_size(746)
> that is caused by an ndr_pull of the text form of a security
> descriptor. We never see this message with the old build system as
> debug.c duplication means it is not printed. If you attach with gdb to
> a Samba built from the old build system you'll see that the error does
> happen, it just doesn't get reported.
> Additionally, with the waf build, we end up with a much smaller set of
> binaries (a bit over 100M, as opposed to about 1.1G with the old
> system). Part of this is the much more extensive use of shared
> libraries, but part of it is because the waf build goes through a
> "object reduction" stage where it removes duplicate object files from
> libraries and binaries. This means these hidden bugs now surface, and
> we have a chance of fixing them.
> The waf based build is certainly not complete yet. For example, I have
> yet to do the compiler hooks to use the Irix and Tru64 compilers, and
> there is work to do on packaging to make is easier for distros to
> build with waf. I also need to document the "wafsamba" API that is
> used to define the SAMBA_BINARY(), SAMBA_LIBRARY() and related
> functions. I have started using pydoctor to produce html documentation
> for those, but it is not anywhere near complete. We also need more
> developer documentationo how to get the most out of the waf build
> I don't think any of these things are reason enough to delay a merge
> into master however, with the proviso that the old build system is
> still the 'reference' build system that we recommend.
> I'd like to thank everyone who has helped get the waf build to this
> point. Quite a few people have done testing, and have suggested useful
> improvements. I've also had a lot of help from Thomas Nagy, the author
> of waf, who has been very generous with his time.
> Cheers, Tridge
Well, the reduction of the size plus undercovering hidden bugs makes for
me a very plus I'm eager to see it in master.
More information about the samba-technical