[distcc] Lock before preprocess?

[Jake McGuire]

On Aug 24, 2004, at 8:03 PM, Martin Pool wrote:
> Oh, OK.  I think I understand your question now.
>
> It locks the volunteer before starting the preprocessor because the
> choice of remote or local determines whether we should run the
> preprocessor or not.  If we decided to run locally we just run the
> whole compiler.

OK, I'll obviously need to take a closer look at the code here.  If we 
decide to run locally (which I don't think we are in our setup; we're 
actually farming out compilation to all of the windows boxes on 
developer's desktops and purposely not compiling on the boxes we use to 
preprocess, link, and run our test environments), why should we be 
locking anything at all?

> Normally running the preprocessor is so cheap compared to running the
> remote compiler that it makes no difference.

Absolutely.  But it's worth considering what happens in abnormal 
conditions, and if we can make them better without impacting nominal 
behavior much, why not?

> When you say "a bunch of client" I take it you mean a bunch of client
> machines, not just a bunch of different make processes on the same
> client machine?

Three different client machines, but a bunch of different make 
processes on each.  All of our developers use distcc to compile a large 
and somewhat poorly laid out codebase.

> I think the real problem there is that they're sharing a single
> distcc_dir, and the locks are not qualified by client name.

But should the locks be qualified by client name?  If a machine is in 
the pool with two slots defined, should every client be able to try to 
connect to it twice?  This seems somewhat backwards, but I guess if the 
thinking is that we'll fall back on local compilation it might not be 
so strange.

> I think the immediate fix for you is to give each client its own
> distcc_dir on a local (possibly tmpfs) filesystem.

Could work.  It's nice being able to define one hosts file, but we 
could probably get around that.