[distcc] load management

[Jake McGuire]

On Apr 27, 2004, at 4:41 PM, Daniel Kegel wrote:
> I'm about to work on something similar but simpler.

Sounds like we can both work on it without stepping on each other too 
badly.

> For starters, I was simply going to have distccd drop incoming
> connections if the load average was over a threshold.  That would
> achieve my limited goal of avoiding killing workstations that
> were busy doing other things (as the client would blacklist the
> busy workstation for a minute).

We're running into issues at my company, possibly related to large 
numbers of users relative to available CPU slots and the one-second 
backoff timer.

> I think I also need to have the client randomize the host list
> on startup.

Yes, without a doubt.

> I don't think blasting all N servers with a UDP packet on
> each distcc run is a good idea.

It depends on the size of your preprocessed file.  The one preprocessed 
file that I looked at from our build was 1.5 megabytes; compared to a 
thousand TCP packets, plus acks, plus setup and teardown, plus sending 
back the object file, a few tens of UDP packets is not a big deal.

Ideally we'd have a proxy server or a hardware loadbalancer, but I 
don't think we have the budget or the network engineering resources for 
that to happen.

> And all my servers are
> identical speed, and probably all running just one job,
> so they're all either busy or not; no connection counts or
> weights to care about.

You'd just run your distcc without load management in that case.

-jake