[ccache] parallel builds with a high cache hit kill the local host

[Martin Pool]

On  7 Jan 2005, Timothee Besset <ttimo at idsoftware.com> wrote:

> I've been using "ccache distcc" for everyday work for quite some
> time now, and it's working great most of the time. However, I'm
> finding that ccache can be trouble with parallel building in some
> circumstances:
> 
> I run between 10 and 15 jobs for a build. I have enough CPUs around
> to distribute that fine with distcc. However, if I wipe out my build
> tree and rebuild, then ccache is going to catch most of the files in
> the build .. and start 15 processes on the local host, making it
> completely unresponsive for a while.

Under Make using -l might help, but SCons doesn't seem to have that.

Running the whole build at nice 15 might help, depending on what
exactly is causing the machine to be unresponsive.

> Is it possible to add a global lock count for ccache so the new
> processes wait for a free work slot before doing their thing? I'm
> not sure if that should be a startup lock, or a lock to check after
> the cache hit/miss decision. A global lock is probably best..

That might be good; it should be pretty easy to add.

If you run ccache underneath distcc (only on localhost) then it should
have this behaviour.

-- 
Martin                                              There is no other window