CMake Proposal

[tridge at samba.org]

Hi Andreas,

 > I've downloaded the latest version of waf and tried to compile the stuff 
 > Tridge migrated. It crashed so I had to use a version Tridge fixed. This 
 > didn't give a good impression of waf on me.

I was using a broken bit of python to extend waf. Since then the
author of waf has looked over what I was doing and pointed out the
mistake. It now works fine with the standard version of waf.

The waf extension language is python. Like any python script, if you
put in bad code it will crash and give a backtrace. One weakness was
that due to the way waf handles exceptions it was often hard to tell
where in your extended code the bug was. The current svn version of
waf has improved exception handling which allows exceptions in your
add-ons to show up correctly. So I've now put the svn version of waf
in my waf-wip branch to make it easier to find bugs in the wafsamba.py
tool.

 > I've checked what compiler they support (gcc, suncc, msvc) and plattforms 
 > (didn't find a list).

platforms for waf itself? It is a python script. It runs wherever
python runs. For Samba4 this is an advantage as we already rely on
python. For Samba3 this would be a new build-time dependency, although
I don't think it is an onerous one.

If you mean what C compilers are supported in the project, then it
should be the same list we support now. It is whatever Samba builds
with.

 > I haven't found a test infrastructure in waf I didn't find information how 
 > they insure that a compiler and a platform is really supported.

I really don't want to change away from our current test
infrastructure. What I want is to have all or exiting test targets
work. In waf, what we would do is add new top level commands. So we
would add "waf test" which runs our existing testsuite. We'll also add
"waf quicktest", "waf testenv" etc.

If you look in source4/selftest/config.mk you'll see how our current
test system is written for our existing build system. The hooks for
waf would be very similar.

I've added a simple example in source4/wscript in my waf-wip tree. It
adds two new top level waf commands,  "test" and "quicktest". The nice
thing about doing it this way is they get added to "waf --help" like
this:

 Main commands (example: ./waf build -j4)
   clean    : removes the build files
   configure: configures the project
   dist     : makes a tarball for redistributing the sources
   distcheck: checks if the sources compile (tarball from 'dist')
   distclean: removes the build directory
   install  : installs the build files
   quicktest: Run the quick test suite
   test     : Run the full test suite
   uninstall: removes the installed files

This is nicer than our current system where users have to know what
our special make targets for testing are. 

 > CMake has a big test infrastructure and they ensure through nightly builds 
 > that cmake works with different compilers on different platforms [1]. I count 
 > 8 different platforms and 7 different compilers. This means
 > 
 > add_library(mylib SHARED mylib.c) will create a shared library on every 
 > platform with different compilers.

knowing how to build a shared library on a particular system is not
hard. Adding a rule that says "use the following flags/commands on
platform FooIX" is not something I'm really worried about. I'm far
more concerned about getting the basic build infrastructure right, and
not losing the good features of our current config.mk system.

 > If I look at waf then I think Samba will end up creating sambawaf maintaining 
 > a build tool and supporting all those platforms and compilers.

The 'wafsamba.py' extension tool to waf that I have written to support
a very large part of our existing build system is shorter than the
CMake rules just for tevent. Anyone who knows python could learn to
maintain it. I'd far rather learn python, which is a generally useful
language, and which is also the language we've chosen for extending
Samba4, then to spend time learning the CMake language.

I think whatever build system we choose we will have to extend it. We
do things in building Samba that nobody else does. The
SUBSYSTEM/MODULE/LIBRARY/BINARY abstraction, and the various scripting
interfaces are unique to Samba. We have to extend the build tool to
support those. 

 > Jelmer asked for cross-compilation support in CMake. I didn't find a note that 
 > waf supports cross-compilation.

It does. See playground/cross/ for a demo of cross-compilation.

I don't think we'll necessarily do it that way though. The way we
cross-compile in Samba is a bit different than the usual approach. We
need to first build the ASN1 and ET compilers using the host compiler,
then we build the rest using the target compiler.

If you look in wafsamba.py you'll see I did this:

    bld.add_group('setup')
    bld.add_group('build_compilers')
    bld.add_group('build_source')
    bld.add_group('prototypes')
    bld.add_group('main')
    bld.add_group('final')

that adds 7 'target groups' for the Samba4 build. That breaks up the
build process into 7 stages (called 'groups'). It won't start on
anything in the next stage until the previous stage is complete.

For building the ASN1 compiler this is the rule I used:

bld.SAMBA_BINARY('asn1_compile',
	'../heimdal/lib/asn1/main.c ../heimdal/lib/asn1/gen.c ../heimdal/lib/asn1/gen_copy.c ../heimdal/lib/asn1/gen_decode.c ../heimdal/lib/asn1/gen_encode.c ../heimdal/lib/asn1/gen_free.c ../heimdal/lib/asn1/gen_glue.c ../heimdal/lib/asn1/gen_length.c ../heimdal/lib/asn1/gen_seq.c ../heimdal/lib/asn1/hash.c ../heimdal/lib/asn1/symbol.c ../heimdal/lib/asn1/asn1parse.c ../heimdal/lib/asn1/lex.c ../heimdal/lib/vers/print_version.c',
	use_hostcc=True,
        cflags='-DSOCKET_WRAPPER_DISABLE=1 -DNSS_WRAPPER_DISABLE=1 -D_SAMBA_HOSTCC_ -I../heimdal_build',
        group='build_compilers',
	deps='HEIMDAL_ROKEN_H replace',
	)

(please excuse the indentation, these rules were auto-generated by my
config.mk to wscript converter, and it doesn't know how to indent
python very well!)

Notice the groups='build_compilers' part? That marks it as part of the
stage where we build the local compilers needed for building
Samba. Also notice the 'use_hostcc=True' option that marks it as
needing the hosts C compiler.

I haven't implemented the code behind use_hostcc yet, but it would
look something like this:

    if use_hostcc:
        compiler=bld.env['HOST_CC']

Strictly speaking we don't need to have all these build stages with
waf, as if we get the dependencies perfect then it will work without
them. I think it is simpler though, as encoding all the dependencies
(especially with auto-prototypes) is hard, which is why Samba4
currently builds with parallel building disabled as we didn't have a
way to ensure that the build order was right. By using stages that
becomes very simple, and we will once again be able to use parallel
builds.

Cheers, Tridge