[distcc] New user / porting

Tom Matelich tmatelich at zetec.com
Thu Feb 27 20:17:39 GMT 2003


From: 'Martin Pool' [mailto:mbp at samba.org]
Sent: Wednesday, February 26, 2003 3:42 PM
> 
> On 26 Feb 2003, Tom Matelich <tmatelich at zetec.com> wrote:
> >
> OK.  Can you please post a bit more context from the logs when you're
> able?
> 
OK, so running with 1.2.2 on HP 11i, with gcc 3.2

I'm doing 2 jobs because that decreases the number of errors.  I'll try to
trim long lines.

DISTCC_HOSTS="localhost zetec11"

>From localhost's distcc.log:
distcc[12131] (dcc_scan_args) scanning arguments: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/main.o main.cpp
distcc[12131] (dcc_scan_args) found object/output file "OBJS/main.o"
distcc[12131] (dcc_scan_args) found input file "main.cpp"
distcc[12131] compile from main.cpp to OBJS/main.o
distcc[12131] (dcc_parse_hosts) found tcp token "zetec11"
distcc[12131] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_localhost_0
distcc[12131] exec on localhost: g++ -c <snip> -o OBJS/main.o main.cpp
distcc[12131] (dcc_spawn_child) forking to execute g++ -c <snip> -o
OBJS/main.o main.cpp
distcc[12132] (dcc_scan_args) scanning arguments: g++ -c <snip> -o
OBJS/meisterwindow.o meisterwindow.cpp
distcc[12132] (dcc_scan_args) found object/output file
"OBJS/meisterwindow.o"
distcc[12132] (dcc_scan_args) found input file "meisterwindow.cpp"
distcc[12132] compile from meisterwindow.cpp to OBJS/meisterwindow.o
distcc[12132] (dcc_parse_hosts) found tcp token "zetec11"
distcc[12131] (dcc_spawn_child) child started as pid12133
distcc[12132] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_localhost_0 failed: Permission denied
distcc[12132] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_0
distcc[12132] (dcc_lock_host) locked
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0
distcc[12132] (dcc_strip_dasho) result: g++ -c <snip> meisterwindow.cpp
distcc[12132] (dcc_spawn_child) forking to execute g++ -E <snip>
meisterwindow.cpp
distcc[12132] (dcc_spawn_child) child started as pid12134
distcc[12132] (dcc_strip_local_args) result: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/meisterwindow.o meisterwindow.cpp
distcc[12132] exec on zetec11: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/meisterwindow.o meisterwindow.cpp
distcc[12132] (dcc_open_socket_out) client got connection to zetec11 port
3632 on fd6
distcc[12132] (dcc_x_token_int) send DIST00000001
distcc[12132] (dcc_x_token_int) send ARGC0000000a
distcc[12132] (dcc_x_token_int) send ARGV00000003
distcc[12132] (dcc_x_token_int) send ARGV00000002
distcc[12132] (dcc_x_token_int) send ARGV00000013
distcc[12132] (dcc_x_token_int) send ARGV00000013
distcc[12132] (dcc_x_token_int) send ARGV00000005
distcc[12132] (dcc_x_token_int) send ARGV00000002
distcc[12132] (dcc_x_token_int) send ARGV00000002
distcc[12132] (dcc_x_token_int) send ARGV00000002
distcc[12132] (dcc_x_token_int) send ARGV00000014
distcc[12132] (dcc_x_token_int) send ARGV00000011
distcc[12406] (dcc_scan_args) scanning arguments: g++ -c <snip> -o
OBJS/meisterwindow.o meisterwindow.cpp
distcc[12406] (dcc_scan_args) found object/output file
"OBJS/meisterwindow.o"
distcc[12406] (dcc_scan_args) found input file "meisterwindow.cpp"
distcc[12406] compile from meisterwindow.cpp to OBJS/meisterwindow.o
distcc[12406] (dcc_parse_hosts) found tcp token "zetec11"
distcc[12406] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_localhost_0
distcc[12406] exec on localhost: g++ -c <snip> -o OBJS/meisterwindow.o
meisterwindow.cpp
distcc[12406] (dcc_spawn_child) forking to execute g++ -c <snip> -o
OBJS/meisterwindow.o meisterwindow.cpp
distcc[12405] (dcc_scan_args) scanning arguments: g++ -c <snip> -o
OBJS/main.o main.cpp
distcc[12405] (dcc_scan_args) found object/output file "OBJS/main.o"
distcc[12405] (dcc_scan_args) found input file "main.cpp"
distcc[12405] compile from main.cpp to OBJS/main.o
distcc[12405] (dcc_parse_hosts) found tcp token "zetec11"
distcc[12406] (dcc_spawn_child) child started as pid12407
distcc[12405] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_localhost_0 failed: Permission denied
distcc[12405] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_0
distcc[12405] (dcc_lock_host) locked
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0
distcc[12405] (dcc_strip_dasho) result: g++ -c <snip> main.cpp
distcc[12405] (dcc_spawn_child) forking to execute g++ <snip> main.cpp
distcc[12405] (dcc_spawn_child) child started as pid12408
distcc[12405] (dcc_strip_local_args) result: g++ -<snip> main.cpp
distcc[12405] exec on zetec11: g++ <snip> -o OBJS/main.o main.cpp
distcc[12405] (dcc_open_socket_out) client got connection to zetec11 port
3632 on fd6

----------------------------------------------------------------------------
------------------
>From zetec11's distccd.log:
distccd[7939] (main) stdin is a tty; assuming --daemon mode
distccd[7939] distccd (version 1.2.2, built Feb 27 2003 09:32:39) listening
on port 3632
distccd[7939] (dcc_ignore_sighup) ignoring SIGHUP
distccd[7940] (dcc_parent_loop) waiting to accept connection
distccd[7940] (dcc_serve_connection) up to 1 children
distccd[7940] (dcc_parent_loop) waiting to accept connection
distccd[7941] (dcc_check_client) connection from 10.13.1.218
distccd[7941] (dcc_setpgid) setpgid(0, 0)
distccd[7941] (dcc_r_token_int) got DIST00000001
distccd[7941] (dcc_r_token_int) got ARGC0000000a
distccd[7941] (dcc_r_argv) reading 10 arguments from job submission
distccd[7941] (dcc_r_token_int) got ARGV00000003
distccd[7941] (dcc_r_argv) argv[0] = "g++"
distccd[7941] (dcc_r_token_int) got ARGV00000002
distccd[7941] (dcc_r_argv) argv[1] = "-c"
distccd[7941] (dcc_r_token_int) got ARGV00000013
distccd[7941] (dcc_r_argv) argv[2] = "-ftemplate-depth-30"
distccd[7941] (dcc_r_token_int) got ARGV00000013
distccd[7941] (dcc_r_argv) argv[3] = "-ftemplate-depth-30"
distccd[7941] (dcc_r_token_int) got ARGV00000005
distccd[7941] (dcc_r_argv) argv[4] = "-Wall"
distccd[7941] (dcc_r_token_int) got ARGV00000002
distccd[7941] (dcc_r_argv) argv[5] = "-W"
distccd[7941] (dcc_r_token_int) got ARGV00000002
distccd[7941] (dcc_r_argv) argv[6] = "-g"
distccd[7941] (dcc_r_token_int) got ARGV00000002
distccd[7941] (dcc_r_argv) argv[7] = "-o"
distccd[7941] (dcc_r_token_int) got ARGV0000000b
distccd[7941] (dcc_r_argv) argv[8] = "OBJS/main.o"
distccd[7941] (dcc_r_token_int) got ARGV00000008
distccd[7941] (dcc_r_argv) argv[9] = "main.cpp"
distccd[7941] (dcc_r_argv) got arguments: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/main.o main.cpp
distccd[7941] (dcc_scan_args) scanning arguments: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/main.o main.cpp
distccd[7941] (dcc_scan_args) found object/output file "OBJS/main.o"
distccd[7941] (dcc_scan_args) found input file "main.cpp"
distccd[7941] compile from main.cpp to OBJS/main.o
distccd[7941] (dcc_accept_job) input file main.cpp, output file OBJS/main.o
distccd[7941] (dcc_r_token_int) got DOTI0018fe02
distccd[7941] (dcc_r_file) received 1637890 bytes to file
/home/karl/local/tmp/distcc_000000d0/server_0000007941.ii
distccd[7941] (dcc_r_file_timed) 1637890 bytes received in 1.026s, rate
1560kB/s
distccd[7941] (dcc_set_input) command before: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/main.o main.cpp
distccd[7941] (dcc_set_input) changed input from "main.cpp" to
"/home/karl/local/tmp/distcc_000000d0/server_0000007941.ii"
distccd[7941] (dcc_set_input) command after: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o OBJS/main.o
/home/karl/local/tmp/distcc_000000d0/server_0000007941.ii
distccd[7941] (dcc_set_output) changed output from "OBJS/main.o" to
"/home/karl/local/tmp/distcc_000000d0/server_0000007941.out"
distccd[7941] (dcc_set_output) command after: g++ -c -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -o
/home/karl/local/tmp/distcc_000000d0/server_0000007941.out
/home/karl/local/tmp/distcc_000000d0/server_0000007941.ii
distccd[7941] (dcc_spawn_child) forking to execute g++ -c
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g -o
/home/karl/local/tmp/distcc_000000d0/server_0000007941.out
/home/karl/local/tmp/distcc_000000d0/server_0000007941.ii
distccd[7941] (dcc_spawn_child) child started as pid7942
distccd[7941] (dcc_collect_child) wait4 was interrupted; retrying
distccd[7941] (dcc_collect_child) child 7942 terminated with status 0
distccd[7941] (dcc_x_token_int) send DONE00000001
distccd[7941] (dcc_x_token_int) send STAT00000000
distccd[7941] (dcc_x_file) send 0 byte file
/home/karl/local/tmp/distcc_000000d0/cc_0000007941.err with token SERR
distccd[7941] (dcc_x_token_int) send SERR00000000
distccd[7941] (dcc_x_file) send 0 byte file
/home/karl/local/tmp/distcc_000000d0/cc_0000007941.out with token SOUT
distccd[7941] (dcc_x_token_int) send SOUT00000000
distccd[7941] (dcc_x_file) send 1398887 byte file
/home/karl/local/tmp/distcc_000000d0/server_0000007941.out with token DOTO
distccd[7941] (dcc_x_token_int) send DOTO00155867
distccd[7941] (dcc_x_file_timed) 1398887 bytes sent in 0.134s, rate
10195kB/s
distccd[7941] (dcc_report_rusage) g++ resource usage: 0.000000s user,
0.000000s system
distccd[7941] g++ on zetec11 completed ok
distccd[7941] (dcc_accept_job) complete; output file: 1398887 bytes
distccd[7940] (dcc_serve_connection) up to 2 children
distccd[7940] (dcc_parent_loop) waiting to accept connection
distccd[7941] (dcc_cleanup_tempfiles) deleted 4 temporary files
distccd[7941] (dcc_exit) exit: code 0; self: 0.0000 user 0.40000 sys;
children: 7.940000 user 0.300000 sys
distccd[7940] (dcc_reap_kids) down to 1 children
distccd[7940] (dcc_reap_kids) Notice: child 7941 exited: exit status 0
distccd[7940] (dcc_parent_loop) waiting to accept connection
----------------------------------------------------------------------------
---------------

With 2 jobs, things seem to work fine (very close to 2x speedup), other than
the lock errors. With more, its harder to tell how things are working.
Running with 4 jobs appeared to be slower than 2, though my benchmarks are
highly flawed, as there are other people remote onto these boxes.  I believe
it farms a little when set to 4 jobs, but then stops.

no farming        -j2               -j4
real     9:13.8   real     4:14.3   real     5:24.6
user     3:30.3   user     1:40.1   user     2:20.7
sys         6.6   sys         4.9   sys        19.2

Here is an unedited first section of the distcc.log with 4 jobs (no further
text below that):

distcc[11756] (dcc_scan_args) scanning arguments: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/meisterdoc.o
meisterdoc.cpp
distcc[11756] (dcc_scan_args) found object/output file "OBJS/meisterdoc.o"
distcc[11756] (dcc_scan_args) found input file "meisterdoc.cpp"
distcc[11756] compile from meisterdoc.cpp to OBJS/meisterdoc.o
distcc[11756] (dcc_parse_hosts) found tcp token "zetec11"
distcc[11757] (dcc_scan_args) scanning arguments: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/treedisplay.o
treedisplay.cpp
distcc[11757] (dcc_scan_args) found object/output file "OBJS/treedisplay.o"
distcc[11757] (dcc_scan_args) found input file "treedisplay.cpp"
distcc[11757] compile from treedisplay.cpp to OBJS/treedisplay.o
distcc[11757] (dcc_parse_hosts) found tcp token "zetec11"
distcc[11754] (dcc_scan_args) scanning arguments: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/main.o main.cpp
distcc[11754] (dcc_scan_args) found object/output file "OBJS/main.o"
distcc[11754] (dcc_scan_args) found input file "main.cpp"
distcc[11754] compile from main.cpp to OBJS/main.o
distcc[11754] (dcc_parse_hosts) found tcp token "zetec11"
distcc[11755] (dcc_scan_args) scanning arguments: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/meisterwindow.o
meisterwindow.cpp
distcc[11755] (dcc_scan_args) found object/output file
"OBJS/meisterwindow.o"
distcc[11755] (dcc_scan_args) found input file "meisterwindow.cpp"
distcc[11755] compile from meisterwindow.cpp to OBJS/meisterwindow.o
distcc[11755] (dcc_parse_hosts) found tcp token "zetec11"
distcc[11756] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_localhost_0
distcc[11756] exec on localhost: g++ -c -DNO_PLT_CALLS -ftemplate-depth-30
-ftemplate-depth-30 -Wall -W -g -DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED
-DQT_NO_ASCII_CAST -DQT_ALTERNATE_QTSMANIP
-I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I. -I..
-I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/meisterdoc.o
meisterdoc.cpp
distcc[11756] (dcc_spawn_child) forking to execute g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ -o OBJS/meisterdoc.o
meisterdoc.cpp
distcc[11754] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_localhost_0 failed: Permission denied
distcc[11755] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_localhost_0 failed: Permission denied
distcc[11757] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_localhost_0 failed: Permission denied
distcc[11754] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_0
distcc[11755] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_0 failed: Permission denied
distcc[11757] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_0 failed: Permission denied
distcc[11754] (dcc_lock_host) locked
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0
distcc[11754] (dcc_strip_dasho) result: g++ -c -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ main.cpp
distcc[11754] (dcc_spawn_child) forking to execute g++ -E -DNO_PLT_CALLS
-ftemplate-depth-30 -ftemplate-depth-30 -Wall -W -g
-DBUILDMEISTER_MAINSTREAM -DDBG_ENABLED -DQT_NO_ASCII_CAST
-DQT_ALTERNATE_QTSMANIP -I/usr/local/eddynet/opt/qt/mkspecs/hpux-g++ -I.
-I.. -I/usr/local/eddynet/include -I/usr/local/eddynet/opt/include
-I/usr/local/eddynet/opt/qt/include -IMOCS/ main.cpp
distcc[11755] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_1
distcc[11757] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_1 failed: Permission denied
distcc[11755] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0 failed: Permission denied
distcc[11755] (dcc_unlock) release lock
distcc[11757] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_2
distcc[11755] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_2 failed: Permission denied
distcc[11757] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0 failed: Permission denied
distcc[11757] (dcc_unlock) release lock
distcc[11755] (dcc_lock_host) locked
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_3
distcc[11757] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_cpu_tcp_zetec11:3632_3 failed: Permission denied
distcc[11757] (dcc_lock_pause) nothing available, sleeping...
distcc[11755] (dcc_lock_host) ERROR: lock
.../distcc_000000dd/lock_xmit_tcp_zetec11:3632_0 failed: Permission denied
distcc[11755] (dcc_unlock) release lock

-----------------------------------------------------------------------
DISCLAIMER:  Information contained in this message and/or
attachment(s) may contain confidential information of Zetec, Inc.
If you have received this transmission in error, please notify
the sender by return email.
-----------------------------------------------------------------------


More information about the distcc mailing list