[distcc] Limiting cpp0, randomizing hosts, more verbose localhost lockfile names

Donohue, Michael mdonohue at paypal.com
Tue Mar 29 19:51:54 GMT 2005


-----Original Message-----
Try using:
$ make -j -l8
on a 4 processor box. This instructs job control to restrict the load to
8
(loosely approximated to 8 jobs wanting to be in a running state
simultaneously). If all of the compilation is done remotely, several
preprocessor tasks can run, hopefully with most blocking on network I/O
allowing full use of system CPU resources with active preprocessing.
-----


I've tried using the load limiting factor in make, but it results in
very bursty behavior.  The problem is that make can fork a LOT of jobs
before they wind up to their full CPU usage.   So, with "-j -l8" I get
about 60 cpp0 processes running almost instantaneously before the load
surpasses 8.  Then, as the processes continue running, the load
continues all the way up to about 20.  Make has stopped spawning jobs,
waiting for the load to come down.   Then, as all the jobs finish, the
load drops to below 8, and make spawns off another 30 jobs before
noticing the load level is too high.

This is pretty much unacceptable for a multi-user developer box.

The preprocessor limiting code I wrote keeps the load at about 4 when I
limit it to 8 preprocessors, on an otherwise unloaded box.   


Michael


More information about the distcc mailing list