[distcc] keeping localhost busy

Jeff rustysawdust at yahoo.com
Fri Sep 26 17:54:42 GMT 2003


Last month there was an interesting thread entitled "distcc over slow net 
links". I have a similar problem in that two of the "animals" on my farm have 
very old CPUs (although they are on a local 100baseT network). On a real farm 
these animals would probably be turned into glue or hamburger, but on a distcc 
farm I think that's animal cruelty! :)

Here's a quote from the original thread posted by Timothee:

> What I'm thinking, is that once local hosts are starved, distcc should
> find out that there is stuff running on slow hosts, and dupe the compile
> work on the local hosts, sending back whatever finishes first.

The thread went into some interesting discussions on how to choose a faster 
machine, and how distcc might be able to keep track of the speeds of each 
server. In the case of this email, I believe "local hosts" meant the hosts on 
his fast local network. But what I'd really like to do is to simply keep 
"localhost" busy. That wouldn't require any of the additional complexity of 
tracking the power of each server, and it also would be a little bit more 
friendly if more than one person was trying to use the farm at a time.

The project I'm working on consists of over 80 libraries, and some of them are 
quite large. As make gets towards the end of each library, I often see those two 
slow machines drag out the end of the build for 20 or 30 seconds. With this many 
libraries, each one of those pauses really starts to hurt. Using distcc's 
excellent new graph tool, it becomes especially obvious when the fast hosts have 
all "scrolled off to white" and you see the two green bars remaining.

I haven't yet looked at the distcc source, so I'm not sure how complex it might 
be to implement a solution to keep the localhost busy. Because I'm not familiar 
with the architecture I'm probably not the best person to design the solution, 
but I'd be more than happy to help try to implement it if there's interest.


-- Jeff





More information about the distcc mailing list