[distcc] gentoo installfest distcc farm

[Nick Rout]

On Wed, 18 Jun 2003 18:04:55 +1000
Martin Pool <mbp at samba.org> wrote:

> On 18 Jun 2003, Nick Rout <nick at rout.co.nz> wrote:
> > Our local LUG is having a small gentoo installfest and intending to use
> > distcc[d] to speed things up. At this stage it seems we will have in the
> > order of 10 machines being installed and up to 20 in the distccd
> > farm.
> 
> Cool.  I hope it goes well.
> 
> Do you have an announcement I can link from the distcc web page?

not yet, its all been on our mailing list

> 
> I haven't ever done this, so I can't give you really firm advice, but
> hopefully I can point you in the right direction..
> 
> > One thing I foresee is people coming and going during the day, so the
> > pool of distccd servers and the pool of installees will not be static.
> > 
> > I have few questions, and welcome any suggestions. 
> > 
> > 1. should we specifically allocate distccd servers to each client (ie 2
> > per installing client), or can we give each client the list of 20 server
> > machines in the hope the load will balance itself out? Having to
> > allocate servers on an individual basis would be a nightmare as people
> > come and go.
> 
> I would be inclined to just share a single pool between all the
> clients.  Run the server in daemon mode so that it can regulate the
> number of accepted jobs.
> 
> Remember to make sure all the machines have the same version of gcc.

does a minor version number matter? gcc3.2.1 vs gcc3.2.2?

> 
> It might be enough to just put the list of hosts in /etc/distcc/hosts,
> and update that as machines come and go.  If one is gone but still in
> the lists then you'll see occasional warning messages, but there
> should be no real problems.
>  

do i put this on each client machine in /etc/distcc/hosts? is that what
you mean (sorry to be thick).

Do you know how emerge copes with a change to the list of ditccd hosts
in the middle of a compile? is /etc/distcc/hosts read once per emerge,
or once per invocation of gcc?

> > 2. alternatively, is there any sort of distccd "master" program? I
> > envisage giving each client the address of the master box which
> > hands the compilation jobs out to the farm. The advantage would be
> > that the master could maintain a single list of servers., which we
> > could change as servers came and went.
> 
> You can use any TCP load balancer, such as "Balance" (balance.sf.net).
> distcc's protocol is very simple.  There are probably 10 other
> balancers listed on freshmeat. 
> 
> In that case you just need to list the IP of the "front end" balancer.
> Remember to crank up the number of jobs allocated to that machine to
> allow for all the CPUs behind it, with something like this in the host
> spec
> 
>   frontend/20
> 
> Please let us know how it turns out.

I will ,and i will look into balance or similar

Thanks for all the help.
> 
> -- 
> Martin 
> 

-- 
Nick Rout <nick at rout.co.nz>