[ccache] parallel builds with a high cache hit kill the local host

[Timothee Besset]

Mhh .. if my compiler is "distcc ccache" .. then I am splitting the cache checks on all the machines, right? Which means I greatly reduce the chance of cache hits really ( specially with 5 hosts involved ). And running ccache on the remote hosts and sharing the pool is a bit too much work than I'm ready to do really.

The machine becomes unresponsive .. in what way .. well .. when you have 15 processes grinding your hard drive and putting the load at 8-10 .. X11 and emacs get sluggish really :)

TTimo

On Sat, 8 Jan 2005 09:23:49 +1100
Martin Pool <mbp at sourcefrog.net> wrote:

> On  7 Jan 2005, Timothee Besset <ttimo at idsoftware.com> wrote:
> 
> > I've been using "ccache distcc" for everyday work for quite some
> > time now, and it's working great most of the time. However, I'm
> > finding that ccache can be trouble with parallel building in some
> > circumstances:
> > 
> > I run between 10 and 15 jobs for a build. I have enough CPUs around
> > to distribute that fine with distcc. However, if I wipe out my build
> > tree and rebuild, then ccache is going to catch most of the files in
> > the build .. and start 15 processes on the local host, making it
> > completely unresponsive for a while.
> 
> Under Make using -l might help, but SCons doesn't seem to have that.
> 
> Running the whole build at nice 15 might help, depending on what
> exactly is causing the machine to be unresponsive.
> 
> > Is it possible to add a global lock count for ccache so the new
> > processes wait for a free work slot before doing their thing? I'm
> > not sure if that should be a startup lock, or a lock to check after
> > the cache hit/miss decision. A global lock is probably best..
> 
> That might be good; it should be pretty easy to add.
> 
> If you run ccache underneath distcc (only on localhost) then it should
> have this behaviour.
> 
> -- 
> Martin                                              There is no other window