[distcc] Re: issue fix proposal for distcc and intel C++ 7.1

[Martin Pool]

On  7 Jul 2003, Markus Werle <numerical.simulation at web.de> wrote:
> Martin Pool wrote:
> 
> > Fallback is enabled by default.
> 
> Yes, something we could discuss.
> 
> Also DISTCC_FALLBACK=1
> AFAICS disables fallback on my platform.

It works for me: if unset, or set to 1, then if the selected server
cannot be reached, distcc runs the compilation locally.  If set to 0,
distcc fails thus:

$ DISTCC_HOSTS=122.222.2.2. DISTCC_VERBOSE=1 DISTCC_FALLBACK=0 ./distcc -c ./cases/hello.c
distcc[16938] (dcc_trace_version) distcc 2.8cvs i686-pc-linux-gnu; built Jul  7 2003 18:35:09
distcc[16938] (dcc_recursion_safeguard) safeguard level=0
distcc[16938] (dcc_trim_path) original PATH /usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
distcc[16938] (dcc_trim_path) not modifying PATH
distcc[16938] (dcc_scan_args) scanning arguments: cc -c ./cases/hello.c
distcc[16938] (dcc_scan_args) found input file "./cases/hello.c"
distcc[16938] (dcc_scan_args) no visible output file, going to add "-o hello.o" at end
distcc[16938] compile from hello.c to hello.o
distcc[16938] (dcc_note_state) note state Startup, file "hello.c", host "(NULL)"
distcc[16938] (dcc_x_token_int) send STFV00000000
distcc[16938] (dcc_x_token_int) send CPID0000422a
distcc[16938] (dcc_x_token_int) send STTE00000007
distcc[16938] (dcc_x_token_int) send FILE00000007
distcc[16938] (dcc_x_token_int) send HOST00000000
distcc[16938] (dcc_get_hostlist) read hosts from environment
distcc[16938] (dcc_parse_hosts) found tcp token "122.222.2.2."
distcc[16938] (dcc_check_backoff) still in backoff period for 122.222.2.2.
distcc[16938] (dcc_remove_disliked) remove 122.222.2.2. from list
distcc[16938] (dcc_build_somewhere) Warning: failed to distribute and fallbacks are disabled
distcc[16938] elapsed compilation time 0.000403s
distcc[16938] (dcc_exit) exit: code 116; self: 0.000000 user 0.000000 sys; children: 0.000000 user 0.000000 sys
distcc[16938] (dcc_cleanup_tempfiles) deleted 0 temporary files

If this is not what you see let me know.

> > Verbose should only be needed for debugging.
> 
> No. there should be several levels of verboseness.
> debugging messages should come with --debug / -d

There is internal support for changing the level of verboseness, and
indeed this can be done when logging to syslog.  At the moment a
binary toggle seems sufficient for the client, because it's either
"just working" and you don't need information, or you're debugging and
you want everything.  But perhaps an intermediate level where you can
just see where jobs are sent would be good.  I suppose this could be
added with DISTCC_VERBOSE=info.

> > I am not sure in what way setting the hostlist on the command line is
> > easier than setting it in a file or the environment.
> 
> Environment vars should be deprecated.
> They make usage unnecessarily complex.

Well, that's your opinion.  At the moment they seem like the best
alternative given the constraints imposed by working with existing
makefiles, etc etc -- see Wayne's mail.

You can trivially transform between the command-line argument syntax
you proposed and environment variables anyhow:

  distcc --hosts='foo bar quux' -c hello.c
  DISTCC_HOSTS='foo bar quux' distcc -c hello.c

Alternatively somebody could write a small script that sets them, as
(I think) ElectricFence does.  Indeed Gentoo ship a script that does
basically that, so from the user's point of view the fact that they
are set in the environment is nearly hidden.

> > I think on the whole I would like to compile a special program called
> > say distcc-icpc, which is bound to that file, or which determines
> > which one to use based on argv[0].
> 
> This sounds like you are avoiding the solution of  the latent problem of
> distcc code: There is no proper command line handling, but there should be
> something like that.

What do you mean?
 
> Did You consider yacc or its alternatives?:

I'm not sure what problem you're trying to solve so considering what
tool to use to solve it is premature.

Do you mean that you would rather express the logic in dcc_scan_args
by a yacc grammar?  

I'm not sure that would be much simpler, since it would still require
the actions to be written in C, and so I don't think it tackles the
heart of the problem, which is that C is not an ideal language for
configuration.

Or do you want to use yacc to read in a configuration file describing
the grammar?

It might be nice if there were some kind of table similar to a popt
configuration, saying:

 -MD           must be local
 -fprofile-arcs must be local
 -o OBJ        set output file
 -o(OBJ)       set output file

etc etc.

The interaction between options means that this can not necessarily be
done by considering each option in isolation.

I think the design challenge is in desiging an appropriate language --
choosing a tool to implement it is a secondary problem.

I think if anybody is going to do the work to generalize it, then it
would be good to make the tool as general as possible, to allow
distribution of not just C compilation, but other tasks.  I hesitate
to add abstraction layers when only one client is understood.
(Perhaps one-and-a-half, if you count icc.)  It would be better to
have an example of another task that can be distributed before
generalizing, to make sure that the generalization is justified and
that it does in fact cover all the necessary factors.

-- 
Martin