[distcc] distcc over slow net links

Timothee Besset ttimo at idsoftware.com
Sun Aug 24 14:01:27 GMT 2003


hey..

I have a LAN distcc setup, which works quite nicely. But now I'd like to
take it to the next level, and take advantage of several server machines
which are sitting on a remote network. I know how to setup things so that
I talk through ssh to those, but I am getting nasty side effects due to
distcc's design:

- the remote machines more than double my local CPU processing power
(hence I'd really like to use them)

- I am on DSL, so talking to those machines is slow. Most of the time is
being spent transferring back and forth.

The transfer time is not a big issue when I am doing a full rebuild from
scratch. But it is when I have only a few modified files, here's what
happens:

- Several compiles start, mostly on the local LAN, and one goes to a remote
machine.
- LAN stuff is done. Build system waits for linking for remote machine.
- remote takes a long time. not specifically compile, but transfer back and forth.

The transfer issue is not a big deal when I rebuild everything, obviously
remote doesn't have time to do as much as the local stuff, but it still
gives me a non-negligible speedup.

What I'm thinking, is that once local hosts are starved, distcc should
find out that there is stuff running on slow hosts, and dupe the compile
work on the local hosts, sending back whatever finishes first.

That's probably a bit tricky to put together .. but it wouldn't be fun
otherwise, right?

TTimo



More information about the distcc mailing list