[distcc] Lock before preprocess?
jake at boom.net
Wed Aug 25 18:29:10 GMT 2004
On Aug 24, 2004, at 8:03 PM, Martin Pool wrote:
> Oh, OK. I think I understand your question now.
> It locks the volunteer before starting the preprocessor because the
> choice of remote or local determines whether we should run the
> preprocessor or not. If we decided to run locally we just run the
> whole compiler.
OK, I'll obviously need to take a closer look at the code here. If we
decide to run locally (which I don't think we are in our setup; we're
actually farming out compilation to all of the windows boxes on
developer's desktops and purposely not compiling on the boxes we use to
preprocess, link, and run our test environments), why should we be
locking anything at all?
> Normally running the preprocessor is so cheap compared to running the
> remote compiler that it makes no difference.
Absolutely. But it's worth considering what happens in abnormal
conditions, and if we can make them better without impacting nominal
behavior much, why not?
> When you say "a bunch of client" I take it you mean a bunch of client
> machines, not just a bunch of different make processes on the same
> client machine?
Three different client machines, but a bunch of different make
processes on each. All of our developers use distcc to compile a large
and somewhat poorly laid out codebase.
> I think the real problem there is that they're sharing a single
> distcc_dir, and the locks are not qualified by client name.
But should the locks be qualified by client name? If a machine is in
the pool with two slots defined, should every client be able to try to
connect to it twice? This seems somewhat backwards, but I guess if the
thinking is that we'll fall back on local compilation it might not be
> I think the immediate fix for you is to give each client its own
> distcc_dir on a local (possibly tmpfs) filesystem.
Could work. It's nice being able to define one hosts file, but we
could probably get around that.
More information about the distcc