[distcc] distcc scalability with # of users?

[Martin Pool]

On 15 Apr 2004, Dan Kegel <dank at kegel.com> wrote:
> Martin Pool wrote:
> >>1. Since the list of hosts read from $prefix/etc/distcc/hosts is
> >>the same for all workstations, every workstation will
> >>issue large compile jobs to itself sometimes even though it'd be better
> >>off only handling preprocessing and linking (right?)
> >
> >Linking counts against jobs running on the local machine too.  If it's
> >linking in parallel with compiling then it should try to do the
> >compiles remotely.
> 
> Ah, so it notices that "localhost" is the same machine as "zytor"
> when running on zytor?

No, it won't do that.

> That's just a direct replacement for the hosts file, though, isn't it?
> I'm not sure I want IT to be involved in this; it's a lot easier for
> me to modify distcc/hosts than it is to create DNS entries!

OK.

I'm not sure then.  Maybe you should have different hosts files for
each machine to randomize theorder.

> >If the workstations have a reasonable amount of memory then running a
> >couple of low-priority daemons should not hurt too much.  Remember it
> >will only accept about 2*NCPUS.
> 
> Yes, but that means the compile jobs (which could run faster on
> some other compile server which is ready and waiting) will execute
> slower.

Yes.

> >We could check the load average before accepting jobs but that is
> >actually a pretty poor measure for modern machines.
> 
> Oh, I dunno, the number of processes in 'R' state seems
> like it'd be a pretty good measure of load if there's
> plenty of RAM and the distcc job wouldn't cause any disk I/O.
> Or the number of processes in 'R' or 'D' state if jobs do tend
> to do disk I/O, maybe.

One problem is that load average responds pretty slowly to changing
state.  It might still be worth trying though.

-- 
Martin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20040416/67b76642/attachment.bin