Samba 4 build system (was: Re: Samba3 build farm can't execute smbtorture4 anymore)

[Jelmer Vernooij]

Am Mittwoch, den 26.12.2007, 14:30 +1100 schrieb Andrew Bartlett:
> On Mon, 2007-12-24 at 02:23 +0100, Jelmer Vernooij wrote:
> > Am Sonntag, den 23.12.2007, 23:01 +0100 schrieb Volker Lendecke:
> > > On Sun, Dec 23, 2007 at 04:54:34PM -0500, simo wrote:
> > > > Then the right way is to work out a way to build it statically, it will
> > > > be useful in general, not fork it out.
> > > > 
> > > > > And to be honest, I don't care if smbtorture is structured
> > > > > nicely, and I don't care about its size. It has to do its
> > > > > job.
> > > > 
> > > > Then fix it, not fork it.
> > > If Jelmer says he is not able to make the Samba4 build
> > > system link that statically, I am certainly not.
> > This is a fundamental flaw in the Samba 4 build system, and not one that
> > can be easily fixed afaik. 
> I had hoped that as the system can determine circular dependencies, it
> could simply proceed as if the second reference to the circle wasn't
> there at all... 
That only works if you're lucky and the build system specifies the
static libraries to the linker in the right order because the linker
searches for symbols only in that order...

It also won't work if there are symbols in subsystem A that depend on
symbols in subsystem B and the other way around. The only way to work
around that is to specify some libraries more than once (which we did
previously by setting LDFLAGS in some places)... There are linker flags
to tell the linker to search all specified libraries for unresolved
symbols, but those flags aren't portable and slow down linking (although
I think the slow down is bearable).

We can't go back to simply specifying the object files when linked as we
did earlier either: we would hit the maximum length of argv[] on some
systems when linking because of the number of object files we include in
a binary. 

> But while I can use our build system, the closest I've got to working on
> it is the auto-generation of .mk files from the asn.1 sources. 
> 
> > The build system looks like a good example of Not-Invented-Here.
> > However, about a year ago, I did an inquiry into the various
> > alternatives that were available at the time and didn't come up with
> > anything better. Most alternatives either require installing another
> > application and are too big to include in Samba or don't care about
> > working on platforms like AIX, Solaris, etc.
> Has this changed now that Python is a reasonable dependency to require?
> (Given the move to python for our internal scripting)
I quite liked scons (written in Python) and it'd probably be small
enough to include. It's not perfect, but easy to understand, portable
and flexible. 

One of the advantages of scons is also that it also uses checksums
rather than just timestamps to determine whether a file is out-of-date.
That means it should be possible to support automatic dependencies and
parallel builds because changing a prototype header doesn't cause
everything to be rebuilt if you didn't change any function signatures.

The main two problems with scons when I looked at it back then were the
fact it was written in Python and the speed. The first would no longer
be a problem. The second has improved, but I'm not sure if it's improved
enough. 

So that's certainly something to have a look at again as well. I wonder
though whether we shouldn't go back to something we know works before we
consider switching to something experimental again. OTOH, it would be a
waste of effort to switch build systems twice in a short period of time.

Cheers and merry christmas,

Jelmer

-- 
Jelmer Vernooij <jelmer at samba.org> - http://samba.org/~jelmer/
Jabber: jelmer at jabber.fsfe.org