[distcc] gentoo installfest distcc farm

[Martin Pool]

On 18 Jun 2003, Nick Rout <nick at rout.co.nz> wrote:
> Our local LUG is having a small gentoo installfest and intending to use
> distcc[d] to speed things up. At this stage it seems we will have in the
> order of 10 machines being installed and up to 20 in the distccd
> farm.

Cool.  I hope it goes well.

Do you have an announcement I can link from the distcc web page?

I haven't ever done this, so I can't give you really firm advice, but
hopefully I can point you in the right direction..

> One thing I foresee is people coming and going during the day, so the
> pool of distccd servers and the pool of installees will not be static.
> 
> I have few questions, and welcome any suggestions. 
> 
> 1. should we specifically allocate distccd servers to each client (ie 2
> per installing client), or can we give each client the list of 20 server
> machines in the hope the load will balance itself out? Having to
> allocate servers on an individual basis would be a nightmare as people
> come and go.

I would be inclined to just share a single pool between all the
clients.  Run the server in daemon mode so that it can regulate the
number of accepted jobs.

Remember to make sure all the machines have the same version of gcc.

It might be enough to just put the list of hosts in /etc/distcc/hosts,
and update that as machines come and go.  If one is gone but still in
the lists then you'll see occasional warning messages, but there
should be no real problems.
 
> 2. alternatively, is there any sort of distccd "master" program? I
> envisage giving each client the address of the master box which
> hands the compilation jobs out to the farm. The advantage would be
> that the master could maintain a single list of servers., which we
> could change as servers came and went.

You can use any TCP load balancer, such as "Balance" (balance.sf.net).
distcc's protocol is very simple.  There are probably 10 other
balancers listed on freshmeat. 

In that case you just need to list the IP of the "front end" balancer.
Remember to crank up the number of jobs allocated to that machine to
allow for all the CPUs behind it, with something like this in the host
spec

  frontend/20

Please let us know how it turns out.

-- 
Martin