[distcc] Multiple people running distcc at once, using one DISTCC_DIR

[Daniel Kegel]

Victor Norman wrote:
> Dan, Jean, et al., 
> 
> I have taken Dan up on his challenge (sort of).  I ran 6 compiles
> simultaneously, each with its own DISTCC_DIR, and with host list randomization
> turned on, and -j20.  What I found is that the compilation times increased
> pretty significantly: from 39 minutes to 60 minutes [ on my 44 node cluster ].

Can you benchmark lower values for -j ?

 > I suspect the results would have been even worse if my modifications
 > weren't included.

Can you try with vanilla distcc-2.16 (with my randomization patch,
http://kegel.com/crosstool/crosstool-0.28-rc34/patches/distcc-2.16/randomhost.patch,
be sure to enable randomization, I think you have to list --randomize as the
first host), and see how much your mods really help?  You might want to
drop slow nodes from the list for this test, not sure.

 > Do you recommend distcc-2.17?

I don't recommend distcc-2.17, as I suspect the longjmp is unsafe.

 > my goal is to have 10 simultaneous compiles (at -j20) work well, so we
 > would have up to 200 requests simultaneously. ...

stock distcc doesn't do well at handling overqueuing.  You may need to
tune your -j value such that # of pending jobs is roughly same as # of CPUs.

 > "The system performance [should] decline gracefully under heavy loads."

You should add "without tuning the -j value" to that requirement, I think.

Making distcc work well even with rediculously high -j values is an
interesting question.  It's possible this is a reasonable thing to
ask for; distcc should throttle back the -j'ness automatically, maybe.

- Dan