[distcc] My big plans for distcc
dank at kegel.com
Fri Sep 3 18:24:09 GMT 2004
Victor Norman wrote:
> o Goals for the system:
> o to pick best servers available, whether 1 engineer is compiling or 20 are.
> o to degrade gracefully under heavy loads.
> o fairness when heavily loaded.
> o to work with compiles dispatched from multiple
> machines simultaneously (i.e., multiple "clients").
How 'bout as a first cut, just picking a random server?
That should meet the above requirements a little better than the current scheme,
especially when the server farm is uniform, and it's cheap.
I've implemented it; the patch is at
> o to work with our heterogeneous network of servers
> -- some very fast, some
> multiprocessor, some very slow, some conditionally
Ixnay on the slow servers. Best way to run a distcc cluster in my experience
is to never use slow machines as servers.
As for hetrogenous servers, I've run clusters that contain linux, MacOSX, and Cygwin
boxes. I did patch distccd a bit, but only because I wanted the
compilers for all three build systems to be visible from all systems,
so I needed distccd to ignore the first couple components of the compiler path.
This is up and running, and is part of the devel version of http://kegel.com/crosstool
> o an "availability server daemon" will run on a
> single machine and will
> write to the hosts file, adding machines that
> become available, and removing machines that are not
> available anymore. And it will write the
> load-averages into the new hosts config file. It
> gets this informationi by periodically receiving
> messages from each machine that is configured to
> participate in the compilation system.
> o All machines in the compilation farm will run a
> small client daemon to
> communicate with this server daemon, so that the
> machine's load-average
> and availability status is updated periodically.
> A machine's
> availability may be determined by its
> load-average, or when its
> screensaver is running or not, etc.
> o we have a prototype for this already -- it is
> basic socket programming,
IMHO that's overkill. However, go ahead and post your code, it might be interesting.
I keep trying to figure out a way to do something like that with it
being useful. Maybe you succeeded.
More information about the distcc