[distcc] load management
Jake McGuire
jake at boom.net
Tue Apr 27 23:04:39 GMT 2004
I am planning to start working on adding load management enhancements
to distcc for use at our company, but wanted to make sure that no one
was almost done fixing the problem and also make sure that I didn't
spend a bunch of time making my changes and have them rejected.
A brief summary of my proposal:
1) have distccd listen on a UDP port for "status requests"
a) when distccd gets a request on this port, send back status info to
the requester
1) number of active compilation requests
2) "server weight"
3) maximum connections
4) other status information TBD
b) forking client
1) just look at dcc_nkids variable
c) preforking client
1) have each child lock/unlock a file when it starts/finishes
compilation
2) distccd tries to lock each file; number of locked files = number
of active connections
3) on startup, distcc determines appropriate server to send request to
a) poll all servers in host list - wait until all respond or 0.1
seconds, whichever comes first
b) multiply connection count by weight, take minimum result, send
request to that server
"least connection" loadbalancing actually does a very good job of
spreading the load around - faster machines process connections faster
and so end up getting more of the work. Weighting is more or less
optional, but can be used if you want some machines in the cluster to
be dedicated compilation boxes and others are also used for other
things.
Thoughts? I'll probably get to this next week or maybe later this week.
-jake
More information about the distcc
mailing list