[distcc] load management

[Daniel Kegel]

Jake McGuire wrote:
> I am planning to start working on adding load management enhancements to 
> distcc for use at our company, but wanted to make sure that no one was 
> almost done fixing the problem and also make sure that I didn't spend a 
> bunch of time making my changes and have them rejected.

I'm about to work on something similar but simpler.

> A brief summary of my proposal:
> 
> 1) have distccd listen on a UDP port for "status requests"
>   a) when distccd gets a request on this port, send back status info to 
> the requester
>     1) number of active compilation requests
>     2) "server weight"
>     3) maximum connections
>     4) other status information TBD
>   b) forking client
>     1) just look at dcc_nkids variable
>   c) preforking client
>     1) have each child lock/unlock a file when it starts/finishes 
> compilation
>     2) distccd tries to lock each file; number of locked files = number 
> of active connections
> 3) on startup, distcc determines appropriate server to send request to
>   a) poll all servers in host list - wait until all respond or 0.1 
> seconds, whichever comes first
>   b) multiply connection count by weight, take minimum result, send 
> request to that server
> 
> "least connection" loadbalancing actually does a very good job of 
> spreading the load around - faster machines process connections faster 
> and so end up getting more of the work.  Weighting is more or less 
> optional, but can be used if you want some machines in the cluster to be 
> dedicated compilation boxes and others are also used for other things.

For starters, I was simply going to have distccd drop incoming
connections if the load average was over a threshold.  That would
achieve my limited goal of avoiding killing workstations that
were busy doing other things (as the client would blacklist the
busy workstation for a minute).
I think I also need to have the client randomize the host list
on startup.

I have more ambitious plans, involving a local proxy server,
but I'm less likely to do those soon.

I don't think blasting all N servers with a UDP packet on
each distcc run is a good idea.  And all my servers are
identical speed, and probably all running just one job,
so they're all either busy or not; no connection counts or
weights to care about.
- Dan