[distcc] load management

[Martin Pool]

On 27 Apr 2004, Jake McGuire <jake at boom.net> wrote:

> We're running into issues at my company, possibly related to large 
> numbers of users relative to available CPU slots and the one-second 
> backoff timer.

Can you expand on this please?

Do you think the client is not finding spare CPU slots that it ought
to run on?

> It depends on the size of your preprocessed file.  The one preprocessed 
> file that I looked at from our build was 1.5 megabytes; compared to a 
> thousand TCP packets, plus acks, plus setup and teardown, plus sending 
> back the object file, a few tens of UDP packets is not a big deal.

I don't think it is a big deal either.  I'm a little more concerned
that it will not map very well to SSH connections, because I kind of
hope that a SSH connection conservation would make it cheap enough to
use in many situations.  It would be a shame if the main load
balancing protocol didn't work with that.  Not necessarily
unacceptable, but kind of a shame.

> Ideally we'd have a proxy server or a hardware loadbalancer, but I 
> don't think we have the budget or the network engineering resources for 
> that to happen.

There is tcpbalance and similar things...  

I think it would be pretty cool if yourself or Dan could try out a
software tcp loadbalancer with distcc.  I think it might work OK:
although the traffic is bursty, the sustained rate should not be so
large as to saturate the loadbalancer machine.  (Just pushing tcp is
pretty cheap.)  

Most of the factors we're considering here like loadaverage and number
of active connections are things that a loadbalancer could do without
needing to know anything about distcc. 

-- 
Martin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20040428/da079d82/attachment.bin