[distcc] distcc over slow net links
Dag Wieers
dag at wieers.com
Mon Aug 25 12:41:36 GMT 2003
On Mon, 25 Aug 2003, Martin Pool wrote:
> On Mon 2003-08-25, Dag Wieers wrote:
>
> > But I like the idea, especially in environments where some of the
> > servers in your cluster are sometimes used for heavy duty. If you're
> > waiting, you might as well bet on another horse (especially when it is
> > at no extra cost)
>
> But there is a cost: other jobs which might arrive in the future or
> which might be sent by another user will not be able to use those
> servers. So we want to do this only when the possible gain is so great
> as to justify the risk.
Well, at the end of compiling the set of objects (either the end of that
parallelization or when -j5 was given, after the 5th job).
Sure there might be a cost, and that's why I would make it optional. And
probably not enabled by default, as the optimized situation is that you
have everything fine-tuned. (That should be expected by distcc as default)
> One can imagine a naive algorithm wasting a lot of time.
>
> To turn the question around: why don't we just schedule the job on the
> nearby machine in the first place?
Because it may make configuring complex (define what machine is nearby
and what not) and a configuration may be static while the environment is
changing a lot (read: I could be moving from offices).
If we could test the latency of the network and the 'power' of each system
before compiling and have a some clever logic for deciding which
systems to use in what order, it would be even better. Because there's no
additional configuring involved and it would work with dynamic set-ups.
The cost however is that the test may take some time (especially with
more machines) and therefor it may not be worthwhile for small
compile-jobs. But that's again up to the user ;)
So it may be nice to have, but I'd personally would rather have the second
implementation (even as default).
-- dag wieers, dag at wieers.com, http://dag.wieers.com/ --
[Any errors in spelling, tact or fact are transmission errors]
More information about the distcc
mailing list