[distcc] Re: plateau on 10Mbps ethernet (was gcc bootstraps with distcc)

[Dara Hazeghi]

--- Martin Pool <mbp at sourcefrog.net> wrote:
> OK.  If I can think of a way to do it that does not
> break
> compatibility too much I'll do it shortly, otherwise
> it might need to
> wait just a little.  (Or perhaps be not on by
> default.)

Not a problem. I found a quick band-aid that I'm
testing now (making gcc emit a comment to the top of
all source files). I just tested Intel cc, Sun cc and
DEC (well, now HP) cc, and all emit stuff for empty
sourcefiles, so it looks like Darwin's in the distinct
minority for now!
 
> If you see many in Connecting, Sending or Receiving
> then that may mean
> that your network is overloaded. 

I'll give that a shot, and report back soon. Given
that I'm also sending assembly, not binary files, I
imagine that'd increase the network hit too. 

> 
> Therefore on 10Mbp/s you can only remotely compile
> about one file per
> second.  If you're building complex programs on
> medium-slow machine
> they might take 4s per file, so at 4 machines or
> less the network will
> be saturated.

I hadn't considered that. I should try compiling KDE
for a change :-) GCC doesn't stress things enough...
> 
> Compression may or may not help there.

That's a good point. I vaguely recall a lzo
compression patch floating around. What did happen
with that?
> 
> Some friends are using distcc on a 1Gb/s network,
> and they benefit
> from up to 24 remote CPUs.  (It's not a linear
> improvement because of
> makefile inefficiencies, but there is an
> improvement.)

Right. I did a -j16 build of the kernel once, and
noticed exactly the same thing (with a faster
network).

Thanks,

Dara

__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com