[distcc] Re: separating compiler and assembler: benchmarks

[Stuart D. Gathman]

Martin Pool wrote:

>>Also, the GNU compiler supports piping preprocessed source into the 
>>compiler via stdin - avoiding having to produce an entire temp file before 
>>beginning the compile.  Similarly, the compiler output goes to stdout, and 
>>can be piped directly to the assembler.  How should this be supported in 
>>distcc?
> 
> distcc used to support something like this but it was disabled because
> of bugs.  In fact, it fed the compiler from a fifo rather than stdin.
> (See the NEWS file.)  I'll look at adding it back in now that there is
> a benchmark suite that can show whether it's faster or not.
> 
> It does allow for more overlapped processing but I'm not sure if it's
> significant.

I have benchmarked my version of remote compile with and without -pipe.  I can 
say that the -pipe option is extremely important.  When properly implemented, 
it cuts the time spent waiting for files to copy over the network in half. 
With -pipe, sending preprocessed source to the volunteer and receiving 
assembler output occur nearly simultaneously.  Without -pipe, the copy in each 
direction must happen sequentially.  Using -pipe does not help distcc 
currently, because the assembler does not (and cannot given the kinds of 
forward reference problems it must solve) incrementally write the object 
output.  However, when the compiler runs on a volunteer, and the assembler 
runs locally, -pipe provides a factor of 2 speedup.  Since the assembler is 
typically fast even on a slow machine, running the assembler locally with 
-pipe is almost always a big win.

Here are some average elapsed times for compiling regex.c with cpp,as running 
on a 100Mhz PPC104 running AIX, and cc1 running on a 1.7Ghz celeron running 
Linux.  The compile was run repeatedly so that everything is cached.

gcc on AIX with no help  52.2 secs
remote cc1, local as without -pipe  9.9 secs
distcc (remote cc1+as, projected)   8.0 secs
remote cc1, local as with -pipe  5.3 secs

The regex.c file is 185530 bytes, regex.o is 37753 bytes, and the regex.s 
output of cc1 is 112049 bytes.  The projected distcc time is based on the time 
to run 'as' locally and the smaller size of regex.o vs regex.s.