[distcc] DistCC / Multicast

[Antenore Gatta]

2006/2/7, Stuart Longland <redhatter at gentoo.org>:
>
> Antenore Gatta **top-posted** -- posts re-ordered and re-formatted for
> readability:
> > 2006/2/7, Martin Pool <mbp at sourcefrog.net>:
> > > On  6 Feb 2006, Antenore Gatta <antenore at gmail.com> wrote:
> > > > Hi Martin,
> > > >
> > > > > distcc uses tcp connections to carry the requests and responses,
> and
> > > > > multicast doesn't work with tcp, only udp.  multicast udp could be
> > > > > good for checking the load of servers or seeing if they have a
> > > > > cached result.
> > >
> > > > Why not? Multicast is very usefull in this situation, all the
> > > > clustered environment are using multicast to join member to the
> > > > cluster.
> > >
> > > Why not use multicast for tcp?  Because as far as I know there's no
> > > specification for how it would work, and no network stack supports it.
> > > :-)
> >
> > Sure, even if there are several research group that are working on new
> > multicast implementation over tcp.
> >
> > http://www.tldp.org/HOWTO/Multicast-HOWTO-9.html
>
> Even if such an implementation were devised, or distcc re-written to
> support sending of packets via UDP multicast... there's still an issue
> of *who* decides to compile the said packet.
>
> If a source packet gets broadcasted to each member of the multicast
> group, what happens... do all of them start compiling that piece of
> source code, and start racing eachother to the finish?  Or do they wait
> random intervals, and one "sticks its hand up" to accept the compile job?
>
> > My explanation was bad, it's due to my orrible English skills... :-|
> >
> > I don't know if it could be a good idea, and therefore this is why I'd
> > like to have yours opinions.
> > It should be possible to write a separate program/utility that check
> > the servers availability using multicast (sure, at udp level).
> > Every servers send messages over multicast to join the group, at this
> > point the client knows exactly how many servers there are and a new
> > configuration file could be written, and the build can begin.
> >
> > In this way you can also have different building farms only using a
> > different multicast group.
> >
> > i.e.
> > 237.0.0.1   5555 1st Compiling farm for i386 architecture
> > 237.0.0.10  4444 2nd Compiling farm for Sparc architecture
> > and so on...
>
> This would be a better, IMHO, way to use UDP multicast; dynamic distcc
> service discovery.  Perhaps it's worth looking into putting support for
> ZeroConf mDNS into a distcc-discovery system?
> --
> Stuart Longland (aka Redhatter)              .'''.
> Gentoo Linux/MIPS Cobalt and Docs Developer  '.'` :
> . . . . . . . . . . . . . . . . . . . . . .   .'.'
> http://dev.gentoo.org/~redhatter             :.'


I don't know almost anything about mDNS and  DNS-SD.
But I think that could be useful.

I'm using distcc in different ways and my great problem is that I need to
change the configuration several times (about 2 times per day on different
compiling farms).
I need a more simple solution, as simple as possible.
Every hosts has always the same IP and I cannot change the network
configuration.
I'd like to use udp only to discover available servers.

This is the scenario:

I've 3 different client and a variable number of servers between 5 to 20.
I'm compiling only 2 times a day using user's available PC. The time frame
windows are from 12:00 to 14:00 and from 18:00 to 21:00.
I want to use only idle desktops, and I don't want users interaction.

My idea is:

1. Screensaver applet like Seti at home that send any 5 minutes a multicast
request to  N different groups until it receives a client response. Sure, it
also could be a command line utility.
2. 1 daemon per client (3 daemon) anyone accepting request on 1 multicast
group.
3. As soon as the daemon reach the minum number of servers, it dinamically
builds the conf file and start the distcc building process.

--- This implementation is also good to use all the desktop's user during
their holidays, without asking anything to anyone ---

Note: User know that they must wait until the end of the build process to
begin to work again.

If you have better ideas please let me know.
-------------- next part --------------
HTML attachment scrubbed and removed