[distcc] distcc over slow net links

Dag Wieers dag at wieers.com
Mon Aug 25 12:41:36 GMT 2003


On Mon, 25 Aug 2003, Martin Pool wrote:

> On Mon 2003-08-25, Dag Wieers wrote:
> 
> > But I like the idea, especially in environments where some of the
> > servers in your cluster are sometimes used for heavy duty. If you're
> > waiting, you might as well bet on another horse (especially when it is
> > at no extra cost)
> 
> But there is a cost: other jobs which might arrive in the future or
> which might be sent by another user will not be able to use those
> servers.  So we want to do this only when the possible gain is so great
> as to justify the risk.  

Well, at the end of compiling the set of objects (either the end of that 
parallelization or when -j5 was given, after the 5th job).

Sure there might be a cost, and that's why I would make it optional. And 
probably not enabled by default, as the optimized situation is that you 
have everything fine-tuned. (That should be expected by distcc as default)


> One can imagine a naive algorithm wasting a lot of time.
> 
> To turn the question around: why don't we just schedule the job on the
> nearby machine in the first place?

Because it may make configuring complex (define what machine is nearby 
and what not) and a configuration may be static while the environment is 
changing a lot (read: I could be moving from offices).

If we could test the latency of the network and the 'power' of each system 
before compiling and have a some clever logic for deciding which 
systems to use in what order, it would be even better. Because there's no 
additional configuring involved and it would work with dynamic set-ups.

The cost however is that the test may take some time (especially with 
more machines) and therefor it may not be worthwhile for small 
compile-jobs. But that's again up to the user ;)

So it may be nice to have, but I'd personally would rather have the second 
implementation (even as default).

--   dag wieers,  dag at wieers.com,  http://dag.wieers.com/   --
[Any errors in spelling, tact or fact are transmission errors]




More information about the distcc mailing list