[PATCH] Consolidate build groups to reduce time taken on single-target builds

Andrew Bartlett abartlet at samba.org
Sun May 26 07:34:31 MDT 2013


On Sun, 2013-05-26 at 14:06 +0200, Volker Lendecke wrote:

> So to summarize, I have a few complaints:
> 
> Long startup time: No idea where the time is spent, but 6
> seconds on a clean build seems really excessive.
> 
> Always compile everything before linking: I'm told that this
> is due to broken dependencies, but I haven't followed this
> exactly. I had thought that the main advantage of waf over
> autoconf is that it is perfect with dependencies, so I
> probably have just not listened right. If waf's way to
> dependency perfection is to compile almost everything
> always, I think that should be doable with autoconf as well.
> 
> Compile individual C files: It seems not possible without a
> gross hack to compile just one file on demand without all
> that long startup phase etc.
> 
> Change CFLAGS between runs: Nobody has commented on this one
> so far.
> 
> The first two are certainly the most annoying ones, but
> probably the two that are hardest to fix. If Tridge failed
> to fix something (startup time), this is my very definition
> of "unfixable". So I can only ask everybody to keep the old
> build system around.

Volker,

I think I may have a solution that should address your first three
complaints, at least in part. 
 
This patch seems to dramatically reduce the time spent on single target
builds.

Groups were added to our waf build for stability during early
development, as dependency information was first imported from the
previous autoconf/perl based build system. 

I don't think we need this distinction in the main build of C files
into .o, and when linking these into binaries, because the invocation of
these tools is very well defined, and we will find any missing inputs
very quickly. 

As such, I've removed the libraries and binaries targets, consolidating
them with 'main'  

By making this change, a build of smbtorture only on a clean tree drops
from 3778 to 2489 targets, and much of the expensive linker stage is
skipped.  The time for a null build of smbtorture only also drops from
4.673s to as low as 2.499s on my laptop.

As your work pattern is to work on a single binary and recompile it,
rather than they entire project, this should benefit you in particular,
and I hope it resolves to some extent your first complaint.  It also
addresses your second complaint, because you are correct we were doing
way to much when asked to compile a single binary.  

While I don't directly assist you with your third issue in being able to
directly compile only 1 C file, by allowing targeting of a single object
this should be at least a little more practical, as only files that the
target binary or library use will be recompiled.

I can't offer a simple solution on the CFLAGS issue, I think I have made
good progress towards your primary concerns, and I hope you can confirm
that these have assisted.  I certainly was surprised to see how much it
helped!

Jelmer/Metze:

Please review/push if appropriate.

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-build-Remove-binaries-and-libraries-build-groups.patch
Type: text/x-patch
Size: 3600 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20130526/45191b3c/attachment.bin>


More information about the samba-technical mailing list