[distcc] Re: issue fix proposal for distcc and intel C++ 7.1

[Markus Werle]

Martin Pool wrote:

> On  7 Jul 2003, Markus Werle <numerical.simulation at web.de> wrote:
> > Martin Pool wrote:
> >
> > > Fallback is enabled by default.
> >
> > Yes, something we could discuss.
> >
> > Also DISTCC_FALLBACK=1
> > AFAICS disables fallback on my platform.
>
> It works for me: if unset, or set to 1, then if the selected server
> cannot be reached, distcc runs the compilation locally.  If set to 0,
> distcc fails thus:
>
> If this is not what you see let me know.

Sorry, my statement is only true for calls like
distcc icpc HelloWorld.C, which gets compiled on the local host
disregarding any environment vars.

For distcc icpc -c HelloWorld.C
the behaviour is fine, but I had expected the call above to also fail,
because I have some very slow (because loaded) machine which is
at least able to link the stuff, but which I dislike to do any compilation.
I assumed setting FALLBACK=0 and removing localhost from the
list of complation hosts would prevent _any_ compilation.


> [...]  I suppose this could be
> added with DISTCC_VERBOSE=info.

Yes, and it can also be controlled by --verbose and --debug like for
many UNIX tools around the world ;-)


> > > I am not sure in what way setting the hostlist on the command line is
> > > easier than setting it in a file or the environment.
> >
> > Environment vars should be deprecated.
> > They make usage unnecessarily complex.
>
> Well, that's your opinion.  At the moment they seem like the best
> alternative given the constraints imposed by working with existing
> makefiles, etc etc -- see Wayne's mail.

As I showed there, a simple proxy script will solve this problem

> You can trivially transform between the command-line argument syntax
> you proposed and environment variables anyhow:
>
>   distcc --hosts='foo bar quux' -c hello.c
>   DISTCC_HOSTS='foo bar quux' distcc -c hello.c

but your solution has side effects on the environment.
Situation remains changed after the run.
This I dislike in general.

> Alternatively somebody could write a small script that sets them, as
> (I think) ElectricFence does.  Indeed Gentoo ship a script that does
> basically that, so from the user's point of view the fact that they
> are set in the environment is nearly hidden.

But Gentoo is a system, a complete environment.
Distcc is some mini tool which should not get married with any system.

> > > I think on the whole I would like to compile a special program called
> > > say distcc-icpc, which is bound to that file, or which determines
> > > which one to use based on argv[0].
> >
> > This sounds like you are avoiding the solution of  the latent problem of
> > distcc code: There is no proper command line handling, but there should be
> > something like that.
>
> What do you mean?

I was kind of provoking you, which was not too nice (sorry).
I kind of dislike the classical C approach with setting pointers and
fiddling with argv[0] ...argv[n].
There is missing some generic solution to command line handling
in distcc. It should take 2 minutes to add some additional command line flag.

AFAIK there _are_ generic solutions to command-line handling
written in C or at least emitting C code, so I took the time to search some of
them and tried to encourage you to pick one and use it.

> > Did You consider yacc or its alternatives?:
>
> I'm not sure what problem you're trying to solve so considering what
> tool to use to solve it is premature.

If you had a generic solution for command-line parsing
you would control every little aspect of your runtime behaviour
via command-line flags.

I have written such a generic tool, and if I get rid of the last little bug
it is ready for a LGPL/GPLed package on sourceforge.
Since the day this tool worked the number of command-line flags of my
main project exploded and my finite-element-code now has a very
comfortable user interface.
The problem for you is that it is written in C++ and that it needs
gcc-3.2 or intel C++ in order to compile.

Q: are you still interested?
In order to not force a redesign of distcc the commandline handler could
set vars in a struct as I proposed earlier. This would mean that only the
commandline handling is written in C++ and the rest of the package could
have access via a struct of char* and stay written in C.


> Do you mean that you would rather express the logic in dcc_scan_args
> by a yacc grammar?
>
> I'm not sure that would be much simpler, since it would still require
> the actions to be written in C, and so I don't think it tackles the
> heart of the problem, which is that C is not an ideal language for
> configuration.

I assumed it was the language of your choice.
I also assumed you were aiming at solving it in that language.
Again, there are many solutions for runtime-configuration and
you could choose one.

> Or do you want to use yacc to read in a configuration file describing
> the grammar?
>
> It might be nice if there were some kind of table similar to a popt
> configuration, saying:
>
>  -MD           must be local
>  -fprofile-arcs must be local
>  -o OBJ        set output file
>  -o(OBJ)       set output file
>
> etc etc.

Yes, that is exactly what my and other tools do.
Hey! popt! I just saw the solution is already at hand.
Use it!

> The interaction between options means that this can not necessarily be
> done by considering each option in isolation.

yes, clashing options can be controlled by many libs

> I think the design challenge is in desiging an appropriate language --
> choosing a tool to implement it is a secondary problem.
>
> I think if anybody is going to do the work to generalize it, then it
> would be good to make the tool as general as possible, to allow
> distribution of not just C compilation, but other tasks.  I hesitate
> to add abstraction layers when only one client is understood.
> (Perhaps one-and-a-half, if you count icc.)

Well, besides the commercial tool from Qt you are nearly alone.
I found one toy package at sourceforge using your code and ccache code
to build a more universal tool, but it looks pretty alpha.

Therefore assume the world needs your tool and that there is not too
much to do to have it support _any_ compiler.

> It would be better to
> have an example of another task that can be distributed before
> generalizing, to make sure that the generalization is justified and
> that it does in fact cover all the necessary factors.

I think introducing a command-line parser is worth the work
even in the absence of a generalization.
I can understand that C is not the language which makes fun
here. Try to see the advantage of total control via combined
command-line and files. Giving the user total control over every
aspect will make the difference.

What I said is a _proposal_.
It is my personal opinion and your mileage may vary.
I respect you decisions.


best regards,


Markus