[distcc] load management

[Martin Pool]

On 27 Apr 2004, Wayne Davison <wayned at samba.org> wrote:
> On Tue, Apr 27, 2004 at 04:04:39PM -0700, Jake McGuire wrote:
> > I am planning to start working on adding load management enhancements 
> > to distcc for use at our company
> 
> Here's an idea I had been kicking around for awhile but never got to.
> All systems that want to make distcc requests need at least one local
> server process.

I think Occam's Razor cuts against this:

 - don't create a new daemon when a file or directory will do 

 - don't use a file or directory when there is a stateless algorithm
   that can give as good behavior

It may be I am wrong and to make it scale well a daemon is needed, but
I would like to be a bit convinced first.

> This server would be responsible for talking with the
> other servers that were around (configured in some manner), keeping
> track of each one's load (they would have either an open socket or get
> regular UDP updates) and also how fast they have executed jobs (per K)
> at various loads.  The idea is that this current status is maintained on
> each local server (so no central server going down is a single point of
> failure), but not by every client when it starts up.  I imagined this
> server being the same one that handles the job requests from other
> machines (if configured to accept remote jobs).

> So, the local distcc executable would just connect to the server at
> localhost, ask it to run a job, and send it the output of the
> preprocessed input.  The local server would be free to either send
> the job off to another host, or run it locally.

I think there are two separable ideas here:

 - Jobs should be dispatched in a way that depends on the load,
   throughput, business and other factors about different machines.

 - This ought to be implemented by having a daemon running on each
   client machine.

I can believe the first one is probably true, but the second does not
seem to necessarily follow.  Possibly you can do it just with a client
state file in DISTCC_DIR.  Possibly there should be a single
network-wide server.

-- 
Martin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20040428/604b2778/attachment.bin