[distcc] Re: issue fix proposal for distcc and intel C++ 7.1

Martin Pool mbp at samba.org
Wed Jul 9 04:49:10 GMT 2003

On  8 Jul 2003, Markus Werle <numerical.simulation at web.de> wrote:

> Sorry, my statement is only true for calls like
> distcc icpc HelloWorld.C, which gets compiled on the local host
> disregarding any environment vars.
> For distcc icpc -c HelloWorld.C
> the behaviour is fine, but I had expected the call above to also fail,
> because I have some very slow (because loaded) machine which is
> at least able to link the stuff, but which I dislike to do any compilation.
> I assumed setting FALLBACK=0 and removing localhost from the
> list of complation hosts would prevent _any_ compilation.

Note that the command you gave, 

  distcc icpc HelloWorld.C

does both compilation and linking.  distcc always links locally, and
must-be-local commands can proceed even with DISTCC_FALLBACK=0, for
two reasons: it's not really a "fallback", and it does not seem useful
to fail them.

I can think of two alternative ways to do this:

1 - distcc could split the command into 

  distcc icpc -o tmp123.o -c HelloWorld.C
  distcc icpc tmp123.o

I don't want to mess with compiler command lines any more than is
really necessary, and in this case I don't think it's justified.  If
you want them separate you can write the Makefile that way.
Compiling and linking in a single command seems to be rare.

2 - It could just refuse to link when fallbacks are disabled, but I
don't think that's very useful.

> Yes, and it can also be controlled by --verbose and --debug like for
> many UNIX tools around the world ;-)

Yes, if distcc were going to handle command line options then those
would make sense.  However, for reasons Wayne already discussed,
except for --help and --version distcc treats all options as being
compiler options, not distcc options.  I think this is the course that
gives best compatibility.  

> Distcc is some mini tool which should not get married with any
> system.

Right, but in general you can write a very simple script that sets up
the right environment.

> I was kind of provoking you, which was not too nice (sorry).
> I kind of dislike the classical C approach with setting pointers and
> fiddling with argv[0] ...argv[n].
> There is missing some generic solution to command line handling
> in distcc. It should take 2 minutes to add some additional command
> line flag.

To me it seems like pointer manipulation is not the main problem.
What we need to decide is, what is the right way for users to specify
which transformations ought to be applied.

At the moment I am leaning towards writing small "skins" which adapt
distcc to a particular compiler by providing a data table describing
the meaning of particular options.

> If you had a generic solution for command-line parsing
> you would control every little aspect of your runtime behaviour
> via command-line flags.

Yes, I agree.  It is just a matter of deciding in which dimensions it
needs to be generic.

For example, we have just discovered that for icpc we need to be able
to add options for the remote compilation, that are not added for cpp
or linking, and that vary depending on the extension of the input
file.  OK, that can be done in principle.  But if we'd rushed to add a
simple data driven system then that kind of subtlety might not be
possible.  (popt cannot do this by itself, for example.)

Now in pseudo-C (or C++) this can be done easily:

    if (!strcmp(name, "icpc) 
        && input_language == LANG_CPP
        && invocation_mode == REMOTE_COMPILE)
        append(remote_options, "-xc++");

But how do you write that in a configuration language?

> I have written such a generic tool, and if I get rid of the last little bug
> it is ready for a LGPL/GPLed package on sourceforge.
> Since the day this tool worked the number of command-line flags of my
> main project exploded and my finite-element-code now has a very
> comfortable user interface.
> The problem for you is that it is written in C++ and that it needs
> gcc-3.2 or intel C++ in order to compile.
> Q: are you still interested?

Yes, I'd be interested to see it.

> Well, besides the commercial tool from Qt you are nearly alone.
> I found one toy package at sourceforge using your code and ccache code
> to build a more universal tool, but it looks pretty alpha.

As a broader example, people in scientific computing use things like
OpenPBS for distributing work.  Perhaps some of them would be better
served by something based on distcc?  


More information about the distcc mailing list