[distcc] feature request: command-line switch to avoid using localhost

[Alexandre Oliva]

I've been working on and off on trying to make distcc useful for
bootstrapping gcc.  The main difficulties are:

- bootstrapping gcc involves building a stage1 compiler, and then
  using that to build stage2, and using stage2 to build stage3.  This
  means the compiler in the build tree must be accessible to distccd.
  I've managed to fix this for the case of a shared NFS filesystem,
  using a wrapper script that transforms relative pathnames into
  network-neutral ones.

- compiling with the integrated preprocessor generates different
  debugging information from what we get with separate preprocessing
  and compilation.  This causes the test at the end of building
  stage3, that compares it with stage2, to fail, rendering the entire
  bootstrap procedure useless (the whole point is to verify that the
  compiler is capable of rebuilding itself, without random changes,
  that always indicate bugs).

  After banging my head against the wall for a couple of days trying
  to fix the differences in debugging output and not only failing, but
  also getting one of the maintainers of the preprocessor to disagree
  it was a bug in the first place :-(, the solution I came up with was
  to arrange for the wrapper script to take out `localhost´ from
  DISTCC_HOSTS, replacing it with the local hostname (which works as
  long as distccd is running on localhost too).  This worked in
  earlier versions of distcc, but it apparently no longer does ;-(


One of my main problems at the moment is that the wrapper script I
created that network-neutralizes pathnames is somewhat slow, such that
using it for small builds such as those of a number of gcc target
libraries doesn't pay off.

So I came up with the idea of reducing the number of times the wrapper
script runs: instead of transforming pathnames for every single
compilation, I get it to print the transformed compiler command (e.g.,
turning ./xgcc -B./ -B/usr/local/.../ into /net/`hostname`/`pwd`/xgcc
-B/net/`hostname`/`pwd`/ -B/usr/local/.../), and then using the second
command all over.

The problem is that, in this case, I can't override the DISTCC_HOSTS
setting, which means I can't take localhost out of it, and then
bootstrap fails.  So I tried to arrange for the script to print say
DISTCC_HOSTS='the edited DISTCC_HOSTS' distcc ..../xgcc ..., but this
failed because the build machinery is too sensitive to quotation
issues, and it ended up setting DISTCC_HOSTS='host1, and then running
the command host2, which didn't quite work as expected.

See, I might set DISTCC_HOSTS without localhost, but this would slow
down the building of stage1 and all the rest of the toolchain that
doesn't undergo bootstrap-checking, so I'm wondering if a command-line
option for distcc that would tell it to not short-circuit localhost
compilations, using distccd running on localhost instead, would be
acceptable.  Failing that, perhaps a separate environment variable?
Even using a different separator for DISTCC_HOSTS would do, as long as
it's not a shell active character.

I take this opportunity to plead people to consider such issues when
proposing additional syntax for DISTCC_HOSTS: if it was possible to
handle DISTCC_HOSTS as a single shell word (perhaps after turning
blanks into say commas), without the risk of any shell active
characters such as {, }, ~, $, quotes getting in the way, outputting
distcc commands that override DISTCC_HOSTS would be far simpler.

Failing all of this, I could use another wrapper script that would
just override localhost in DISTCC_HOSTS, but this would certainly not
be as fast.

I'm willing to implement one of the changes in distcc that I proposed
above, but I'd like to avoid wasting time implementing something that
wouldn't be acceptable, so I'm asking first.  Which, if any, would you
prefer to have in distcc?

Thanks,

-- 
Alexandre Oliva   Enjoy Guarana', see http://www.ic.unicamp.br/~oliva/
Red Hat GCC Developer                 aoliva@{redhat.com, gcc.gnu.org}
CS PhD student at IC-Unicamp        oliva@{lsd.ic.unicamp.br, gnu.org}
Free Software Evangelist                Professional serial bug killer