[distcc] Re: issue fix proposal for distcc and intel C++ 7.1

[Markus Werle]

Martin Pool wrote:

> > Yes, and it can also be controlled by --verbose and --debug like for
> > many UNIX tools around the world ;-)
>
> Yes, if distcc were going to handle command line options then those
> would make sense.  However, for reasons Wayne already discussed,
> except for --help and --version distcc treats all options as being
> compiler options, not distcc options.  I think this is the course that
> gives best compatibility.

I already showed in this thread that it is quite simple to
just use the convention that only args after
--compiler=<somecompiler> are treated as compiler options.

The reasons Wayne discussed can all be resolved easily except one,
for which I propose a workaround.

Wayne states

> a space-containing $CC value does not always
> work with all projects, and any $CC value at all may not be honored by
> some build setups.  Thus, for me, I never set CC and just add a distcc
> masquerade dir to my PATH (along with ccache):

>    PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:$PATH" make

I consider such a build system as exception (or broken system)
So I accept that these systems need a workaround, but I disagree that the
workaround should affect "normal" systems like simple linux+make.
I still think that it is best practice to use command-line reading for control.

I proposed to Wayne to use wrapper scripts in such cases - which is
exactly the way you are just thinking about when you say:

> At the moment I am leaning towards writing small "skins" which adapt
> distcc to a particular compiler by providing a data table describing
> the meaning of particular options.

Wayne refused the scripts I proposed, saying that

> That's a bad way to go because (1) you have to have a separate set of
> fake-compiler scripts for every user to set their own options,  (2) you'd
>  need to script PATH-munging or use an absolute PATH to the compiler
>  (requiring the compiler to be in the same bin dir on all machines in the
>  latter case),
> (3) to change the options would require you to edit/change
>  all the scripts, and (4) it's slightly less efficient (with an extra
>  shell exec before distcc runs).  No thanks

How about this:
The script for the broken systems reads in all those environment vars
and translates them to runtime args for distcc, while distcc
itself gets rid of environment dependency and uses command-line parsing
as proposed for unbroken systems.

Like this we get maximum flexibility for the unbroken _and_ the broken systems:
If on system foo compiler bar needs some other flag to achieve -xc++,
e.g. it needs -Xc++, this is no longer a problem:

On unbroken system we call
make CXX="distcc --extra-remote-cflags=\"-Xc++\" --compiler=bar -O2 -g HelloWorld.C"

while on broken systems we rely on a symbolic link
/opt/distcc-2.9.1/bin/cc -> /opt/distcc-2.9.1/bin/wrap_distcc.sh

and use

export DISTCC_EXTRA_REMOTE_FLAGS=-Xc++
export DISTCC_COMPILER=bar
export PATH=/opt/distcc-2.9.1/bin:$PATH

make

Did I miss something?


> [...]

> Yes, I agree.  It is just a matter of deciding in which dimensions it
> needs to be generic.
>
> For example, we have just discovered that for icpc we need to be able
> to add options for the remote compilation,

> [...]
> But how do you write that in a configuration language?

This is why I propose command-line flag  --extra-remote-cflags

> > I have written such a generic tool,
> > The problem for you is that it is written in C++ and that it needs
> > gcc-3.2 or intel C++ in order to compile.
> >
> > Q: are you still interested?
>
> Yes, I'd be interested to see it.

I make a package out of it. May take some time due to
much work here.

> > Well, besides the commercial tool from Qt you are nearly alone.
> > I found one toy package at sourceforge using your code and ccache code
> > to build a more universal tool, but it looks pretty alpha.
>
> As a broader example, people in scientific computing use things like
> OpenPBS for distributing work.  Perhaps some of them would be better
> served by something based on distcc?

I don't know.


Markus