[distcc] Load balancing

Tim Small tim at digitalbrain.com
Wed Sep 11 12:57:01 GMT 2002

Martin Pool wrote:

>I'm not saying it's impossible, but it seems quite hard.  If you
>invent a better load spreading algorithm of course I would be happy to
>hear of it.  Having a central controller is a more complex design, so
>I don't want to use it unless it will clearly perform better.
Mmm, we've chosen to use LVS for this, as I've previously said, and I 
think this is probably a good generic way to solve this sort of problem 
- the loadbalancing overhead is extemely low - the most efficient mode 
of operation just changes the MAC address of inbound packets so that TCP 
connection x packets are steered to machine y (the replies go directly 
from distcc server to distcc client, missing out the central server), 
all work is done in kernel mode, so it should scale very well for use a 
large sites.

We have a couple of perl scripts, which we use in conjunction with mon, 
and LVS to move boxes in/out of the build cluster, and we are thinking 
of extending this to modify the individual machine weightings (i.e. new 
job priority), depending on such factors as load, cpu usage etc..

I would think we'll get into all sorts of dynamic load balancing issues 
such as setting up oscilations in load, plus the problem adding 
additional load to the system as a whole.  Perhaps perl or python would 
be good choices for this component, as the work isn't time critical, and 
both have snmp client libraries (prob best way of checking the status of 
machines in the cluster) - also it should be possible to do all of this 
work independently of distcc (thus both keeping distcc simple for the 
simple usage case, and making the system generic so it can by applied to 
other solutions which use LVS, such as web serving).


More information about the distcc mailing list