[distcc] distcc problem
Mattijs Janssens
m.janssens at opencfd.co.uk
Thu Jul 27 15:37:00 GMT 2006
Problem: distcc does not compile correctly. It produces really funny error
messages from the compiler. I switched on logging and cannot see any
problems. The only problems I can see:
- the path to the compiler is quite long. Might overrun some buffer?
- the path to the compiler contains a soft link. The actual file isn't. Might
upset masquerading?
- if I do all the steps by hand that distcc does (i.e. separate -E step) I
cannot reproduce the compiler problems. I searched for all the possible g++
executables on the machine but cannot find anyone that produces below error
messages.
Here's a full report:
1. distcc version 2.18.3 (built from sources):
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --enable-threads=posix --prefix=/usr
--with-local-prefix=/usr/local --infodir=/usr/share/info
--mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64
--enable-languages=c,c++,objc,fortran,java,ada --enable-checking=release
--with-gxx-include-dir=/usr/include/c++/4.1.0 --enable-ssp --disable-libssp
--enable-java-awt=gtk --enable-gtk-cairo --disable-libjava-multilib
--with-slibdir=/lib64 --with-system-zlib --enable-shared
--enable-__cxa_atexit --enable-libstdcxx-allocator=new
--without-system-libunwind --with-cpu=generic --host=x86_64-suse-linux
Thread model: posix
gcc version 4.1.0 (SUSE Linux)
2. Platform: Suse 10.0 (64 bit) with gcc-4.1.0
3. Trying to use distcc to compile C++ library OpenFOAM
4. It compiled but generated a whole lot of error messages I do not get when
compiling local. Error messages are of the form:
/usr/include/pthread.h:63: error: stray ‘#’ in program
In file included
from /home/hunt2/mattijs/OpenFOAM/linuxAMD64/gcc-4.1.1/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.1.1/../../../../include/c++/4.1.1/x86_64-unknown-linux-gnu/bits/gthr-default.h:44,
5. Invocation of distcc:
distcc g++ -m32 -Dlinux -DDP -Wall -Wno-strict-aliasing -Wextra
-Wno-unused-parameter -Wold-style-cast -O3 -DNoRepository
-ftemplate-depth-30
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/triSurface/lnInclude
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/meshTools/lnInclude
-IlnInclude -I.
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/OpenFOAM/lnInclude -fPIC
-pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C -o
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
6. Client logging (copied from stdout):
distcc[12879] (dcc_trace_version) distcc 2.18.3 x86_64-unknown-linux-gnu;
built Jul 25 2006 17:27:24
distcc[12879] (dcc_recursion_safeguard) safeguard level=0
distcc[12879] (main) compiler name is "distcc"
distcc[12879] (dcc_scan_args) scanning arguments: g++ -m32 -Dlinux -DDP -Wall
-Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-DNoRepository -ftemplate-depth-30
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/triSurface/lnInclude
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/meshTools/lnInclude
-IlnInclude -I.
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/OpenFOAM/lnInclude -fPIC
-pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C -o
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
distcc[12879] (dcc_scan_args) found input file
"fvMesh/fvBoundaryMesh/fvBoundaryMesh.C"
distcc[12879] (dcc_scan_args) found object/output file
"Make/linuxGcc4DPOpt/fvBoundaryMesh.o"
distcc[12879] compile from fvBoundaryMesh.C to fvBoundaryMesh.o
distcc[12879] (dcc_get_hostlist) read hosts from environment
distcc[12879] (dcc_parse_hosts) found tcp token "10.0.0.201:3633"
distcc[12879] (dcc_lock_host) got cpu lock on 10.0.0.201:3633 slot 0 as fd3
distcc[12879] (dcc_strip_dasho) result: g++ -m32 -Dlinux -DDP -Wall
-Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-DNoRepository -ftemplate-depth-30
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/triSurface/lnInclude
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/meshTools/lnInclude
-IlnInclude -I.
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/OpenFOAM/lnInclude -fPIC
-pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C
distcc[12879] (dcc_spawn_child) forking to execute: g++ -m32 -Dlinux -DDP
-Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-DNoRepository -ftemplate-depth-30
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/triSurface/lnInclude
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/meshTools/lnInclude
-IlnInclude -I.
-I/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/OpenFOAM/lnInclude -fPIC
-pthread -E fvMesh/fvBoundaryMesh/fvBoundaryMesh.C
distcc[12880] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12879] (dcc_spawn_child) child started as pid12880
distcc[12879] (dcc_strip_local_args) result: g++ -m32 -Wall
-Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-ftemplate-depth-30 -fPIC -pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C
-o Make/linuxGcc4DPOpt/fvBoundaryMesh.o
distcc[12879] exec on 10.0.0.201:3633: g++ -m32 -Wall -Wno-strict-aliasing
-Wextra -Wno-unused-parameter -Wold-style-cast -O3 -ftemplate-depth-30 -fPIC
-pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C -o
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
distcc[12879] (dcc_note_state) note state 2, file "fvBoundaryMesh.C", host
"10.0.0.201"
distcc[12879] (dcc_connect_by_addr) started connecting to 10.0.0.201:3633
distcc[12879] (dcc_select_for_write) select for write on fd4
distcc[12879] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc[12879] (dcc_x_token_int) send DIST00000001
distcc[12879] (dcc_x_token_int) send ARGC0000000f
distcc[12879] (dcc_x_token_int) send ARGV00000003
distcc[12879] (dcc_x_token_int) send ARGV00000004
distcc[12879] (dcc_x_token_int) send ARGV00000005
distcc[12879] (dcc_x_token_int) send ARGV00000014
distcc[12879] (dcc_x_token_int) send ARGV00000007
distcc[12879] (dcc_x_token_int) send ARGV00000015
distcc[12879] (dcc_x_token_int) send ARGV00000010
distcc[12879] (dcc_x_token_int) send ARGV00000003
distcc[12879] (dcc_x_token_int) send ARGV00000013
distcc[12879] (dcc_x_token_int) send ARGV00000005
distcc[12879] (dcc_x_token_int) send ARGV00000008
distcc[12879] (dcc_x_token_int) send ARGV00000002
distcc[12879] (dcc_x_token_int) send ARGV00000026
distcc[12879] (dcc_x_token_int) send ARGV00000002
distcc[12879] (dcc_x_token_int) send ARGV00000024
distcc[12879] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc[12879] (dcc_collect_child) cpp child 12880 terminated with status 0
distcc[12879] (dcc_collect_child) cpp times: user 0.148009s, system 0.048003s,
2064 minflt, 0 majflt
distcc[12879] cpp fvMesh/fvBoundaryMesh/fvBoundaryMesh.C on localhost
completed ok
distcc[12879] (dcc_x_file) send 2278124 byte file /tmp/distcc_a9f2ac03.ii with
token DOTI
distcc[12879] (dcc_x_token_int) send DOTI0022c2ec
distcc[12879] (dcc_pump_sendfile) Notice: sendfile: partial transmission of
65130 bytes; retrying 2212994 @0
distcc[12879] (dcc_select_for_write) select for write on fd4
distcc[12879] (dcc_pump_sendfile) select() returned, continuing to write
distcc[12879] (dcc_pump_sendfile) Notice: sendfile: partial transmission of
114688 bytes; retrying 2098306 @0
distcc[12879] (dcc_select_for_write) select for write on fd4
distcc[12879] (dcc_pump_sendfile) select() returned, continuing to write
etc.
300sdistcc[12879] (dcc_r_token_int) got DONE00000001
distcc[12879] (dcc_r_result_header) got response headerdistcc[12879]
(dcc_note_state) note state 6, file "(NULL)", host "(NULL)"
distcc[12879] (dcc_r_token_int) got STAT00000100distcc[12879]
(dcc_r_token_int) got SERR004cf665
and then I get all these error messages which are e.g. mostly of the form:
/usr/include/stdio.h:374: error: stray ‘#’ in program
7. Server logging:
Jul 27 13:05:16 borg1 distccd[12862]: (main) chdir to /tmp
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_setup_daemon_path) daemon's PATH
is /home/hunt2/mattijs/OpenFOAM/linux/paraview-2.4.4/bin:/home/hunt2/mattijs/OpenFOAM/linux/cmake-2.0.3/bin:/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/lam-7.1.1/platforms/linuxGcc4DPOpt/bin:/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/src/mico-2.3.12/platforms/linuxGcc4DPOpt/bin:/home/hunt2/mattijs/OpenFOAM/linux/j2sdk1.4.2_05/bin:/home/hunt2/mattijs/OpenFOAM/linux/gcc-4.1.1-64/bin:/home/hunt2/mattijs/OpenFOAM/mattijs-1.3.2/applications/bin/linuxGcc4DPOpt:/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/applications/bin/linuxGcc4DPOpt:/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/wmake:/home/hunt2/mattijs/OpenFOAM/OpenFOAM-1.3.2/bin:.:/home/hunt2/mattijs/bin:/home/hunt2/mattijs/bin/DEBUG:/home/hunt2/mattijs/bin/ftpStuff:/home/hunt2/mattijs/envs/Linux/local/bin:/home/hunt2/mattijs/s1studio/ce/bin:/usr/local/bin:/home/hunt2/mattijs/pub/GTS/install/bin:/sbin:/home/hunt2/mattijs/pub/ac3dlx-4.0:/home/hunt2/mattijs/pub/tetgen1.3.4:/usr/X11R6/bin:/usr/bin:/bin:/usr/sbin
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_listen_by_addr) listening on
0.0.0.0:3633
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_defer_accept) TCP_DEFER_ACCEPT
turned on
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_standalone_server) 1 CPU online on
this server
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_standalone_server) allowing up to 1
active jobs
Jul 27 13:05:16 borg1 distccd[12862]: (dcc_ignore_sighup) ignoring SIGHUP
Jul 27 13:05:16 borg1 distccd[12866]: (dcc_detach) setsid to session 12866
Jul 27 13:05:16 borg1 distccd[12866]: (dcc_log_daemon_started) preforking
daemon started (2.18.3 x86_64-unknown-linux-gnu, built Jul 25 2006 17:27:26)
Jul 27 13:05:16 borg1 distccd[12866]: (dcc_preforking_parent) up to 1 children
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_check_client) connection from
10.0.0.201:46992
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_check_address) match client
0xc900000a, value 0xc900000a, mask 0xffffffff
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got DIST00000001
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGC0000000f
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) reading 15 arguments from
job submission
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000003
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[0] = "g++"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000004
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[1] = "-m32"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000005
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[2] = "-Wall"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000014
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[3] =
"-Wno-strict-aliasing"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000007
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[4] = "-Wextra"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000015
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[5] =
"-Wno-unused-parameter"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000010
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[6] =
"-Wold-style-cast"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000003
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[7] = "-O3"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000013
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[8] =
"-ftemplate-depth-30"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000005
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[9] = "-fPIC"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000008
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[10] = "-pthread"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000002
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[11] = "-c"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000026
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[12] =
"fvMesh/fvBoundaryMesh/fvBoundaryMesh.C"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000002
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[13] = "-o"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got ARGV00000024
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) argv[14] =
"Make/linuxGcc4DPOpt/fvBoundaryMesh.o"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_argv) got arguments: g++ -m32
-Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-ftemplate-depth-30 -fPIC -pthread -c fvMesh/fvBoundaryMesh/fvBoundaryMesh.C
-o Make/linuxGcc4DPOpt/fvBoundaryMesh.o
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_scan_args) scanning arguments: g++
-m32 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter
-Wold-style-cast -O3 -ftemplate-depth-30 -fPIC -pthread -c
fvMesh/fvBoundaryMesh/fvBoundaryMesh.C -o
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_scan_args) found input file
"fvMesh/fvBoundaryMesh/fvBoundaryMesh.C"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_scan_args) found object/output file
"Make/linuxGcc4DPOpt/fvBoundaryMesh.o"
Jul 27 13:05:23 borg1 distccd[12867]: compile from fvBoundaryMesh.C to
fvBoundaryMesh.o
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_run_job) output file
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_input_tmpnam) input file
fvMesh/fvBoundaryMesh/fvBoundaryMesh.C
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_token_int) got DOTI0022c2ec
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_file) received 2278124 bytes to
file /tmp/distccd_7fa5ac03.ii
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_r_file_timed) 2278124 bytes
received in 0.005931s, rate 375102kB/s
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_set_input) changed input from
"fvMesh/fvBoundaryMesh/fvBoundaryMesh.C" to "/tmp/distccd_7fa5ac03.ii"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_set_input) command after: g++ -m32
-Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-ftemplate-depth-30 -fPIC -pthread -c /tmp/distccd_7fa5ac03.ii -o
Make/linuxGcc4DPOpt/fvBoundaryMesh.o
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_set_output) changed output from
"Make/linuxGcc4DPOpt/fvBoundaryMesh.o" to "/tmp/distccd_7fc4ac03.o"
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_set_output) command after: g++ -m32
-Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -O3
-ftemplate-depth-30 -fPIC -pthread -c /tmp/distccd_7fa5ac03.ii
-o /tmp/distccd_7fc4ac03.o
Jul 27 13:05:23 borg1 distccd[12867]:
(dcc_check_compiler_masq) /home/hunt2/mattijs/OpenFOAM/linux/gcc-4.1.1-64/bin/g++
is not a symlink
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_spawn_child) forking to execute:
g++ -m32 -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter
-Wold-style-cast -O3 -ftemplate-depth-30 -fPIC -pthread
-c /tmp/distccd_7fa5ac03.ii -o /tmp/distccd_7fc4ac03.o
Jul 27 13:05:23 borg1 distccd[12882]: (dcc_increment_safeguard) setting
safeguard: _DISTCC_SAFEGUARD=1
Jul 27 13:05:23 borg1 distccd[12867]: (dcc_spawn_child) child started as
pid12882
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_collect_child) cc child 12882
terminated with status 0x100
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_collect_child) cc times: user
1.004062s, system 0.300018s, 19636 minflt, 0 majflt
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_x_token_int) send DONE00000001
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_x_token_int) send STAT00000100
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_x_file) send 5043813 byte
file /tmp/distcc_c8d3ac03.stderr with token SERR
Jul 27 13:05:25 borg1 distccd[12867]: (dcc_x_token_int) send SERR004cf665
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_pump_sendfile) Notice: sendfile:
partial transmission of 393180 bytes; retrying 4650633 @0
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_pump_sendfile) ERROR: sendfile
failed: Connection reset by peer
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_x_file) send 0 byte
file /tmp/distcc_898aac03.stdout with token SOUT
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_x_token_int) send SOUT00000000
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_writex) ERROR: failed to write:
Broken pipe
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_x_token_int) send DOTO00000000
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_writex) ERROR: failed to write:
Broken pipe
Jul 27 13:05:26 borg1 distccd[12867]: g++
fvMesh/fvBoundaryMesh/fvBoundaryMesh.C on localhost failed
Jul 27 13:05:26 borg1 distccd[12867]: job complete
Jul 27 13:05:26 borg1 distccd[12867]: (dcc_cleanup_tempfiles) deleted 4
temporary files
--
Mattijs Janssens
OpenCFD Ltd.
The Mews, Picketts Lodge,
Picketts Lane, Salfords,
Surrey RH1 5RG.
Tel: +44 (0)1293 821272
Email: M.Janssens at OpenCFD.co.uk
URL: http://www.OpenCFD.co.uk
More information about the distcc
mailing list