[distcc] DistCC / Multicast

[Dan Kegel]

On 2/6/06, Stuart Longland <redhatter at gentoo.org> wrote:
> One problem I foresee, is if you have machines on the same network with
> differing CPU architectures and toolchain versions.
>
> To compile MIPS n32 binaries, I need a toolchain set up for the
> mips64-unknown-linux-gnu- CHOST.  The distcc daemon needs to figure out
> what toolchains it has -- a i?86-pc-linux-gnu toolchain will not build
> MIPS binaries.  AMD64 vs x86 has a similar issue, x86 toolchains will
> not produce AMD64 binaries.

Aw, just use crosstool to build all your toolchains,
then you can have every toolchain installed everywhere.
That's what I do :-)

> Also, scheduling would be an issue.  Certainly some sort of host
> discovery would be a nice feature (a `distcc-probe` facility, if you
> will), but it'd need to take into account tool versions and targets in
> order to work properly.

Any host discovery feature should accept a string indicating the
type of server you want to discover, I think.

And it should also make it impossible for an intruder to
sneak a fake server onto the list.  (In other words, it
should be easy but not *too easy* to add a new distcc server!)

And it should avoid introducing a single point of failure.  distcc
lacks a central server that could fail, and it'd be nice to preserve
that feature.

I'm currently using a host discovery procedure that meets
the above criteria.  I've posted about it several times, so this is
nothing new, but I'll describe it again for those who haven't already heard:

To play, your site has to have a DNS server for its intranet.
All the distcc servers to be discovered must be listed in DNS
according to a naming scheme like mydistcc1, mydistcc2, mydistcc3, ...
The client's Makefile or build script starts by running 'lsdistcc mydistcc%d'
lsdistcc looks in DNS for %d=1, 2, 3, ... until it has enumerated all
the servers,
then sends a test compile job to each server (all in parallel).  The servers
that successfully compile the ten-line file in under a second are
considered 'good'.
The output of that tool is a list of good distcc servers.

lsdistcc was surprisingly hard to write because it has to be blazingly fast
and handle network dropouts robustly.
It's been in heavy use here for several months now.
Not only does it ease administration (no more host list files), but it
also protects you against stuck or way-overloaded servers (which are
all too common in the real world).  Large sites really shouldn't try using
distcc without something like it, IMHO.

I'll post a copy of it along with the server-side caching code once
that's ready.
- Dan

--
Wine for Windows ISVs: http://kegel.com/wine/isv