[distcc] distcc scalability with # of users?

Martin Pool mbp at samba.org
Fri Apr 16 04:22:19 GMT 2004

On 15 Apr 2004, Dan Kegel <dank at kegel.com> wrote:
> I'm interested in using distcc in a group of 16 programmers
> where all 16 workstations are equally fast, all 16
> workstations are both compile servers and compile clients,
> and all access a shared copy of distcc over NFS or SMB.
> Several potential problems come to mind when thinking about this:

> 1. Since the list of hosts read from $prefix/etc/distcc/hosts is
> the same for all workstations, every workstation will
> issue large compile jobs to itself sometimes even though it'd be better
> off only handling preprocessing and linking (right?)

Linking counts against jobs running on the local machine too.  If it's
linking in parallel with compiling then it should try to do the
compiles remotely.

More of a problem is that most makefiles will do all the links at the
end in an unparallelized way.

It would probably be good to finish off support for DNS multi-A
records, and use that to spread work across machines.  I don't think
much more needs to be done.

> 2. Distcc won't currently check the load average of each compile server,
> so workstations busy with non-distcc jobs will get slammed with
> distcc jobs, negatively impacting normal use of the workstations.

If the workstations have a reasonable amount of memory then running a
couple of low-priority daemons should not hurt too much.  Remember it
will only accept about 2*NCPUS.

We could check the load average before accepting jobs but that is
actually a pretty poor measure for modern machines.

> 3. If more than one user is issuing distcc jobs, their distcc's
> will sometimes issue jobs to the same machine by chance
> (fairly often, if distcc assigns jobs in order of the etc/distcc/hosts 
> file).

Right, so those jobs will just stall for a bit.

> Has anyone looked at these issues?   I suppose a first step I
> might take if nobody else has might be to run a few benchmarks
> to see if these potential problems actually happen in the real
> world.

That would be good.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20040416/cbbaf851/attachment.bin

More information about the distcc mailing list