[distcc] (fwd from s.wilson@eris.qinetiq.com) Problems with distcc cross-compilation: Sharp Zaurus <-> Linux...
Martin Pool
mbp at samba.org
Mon Nov 4 19:21:04 GMT 2002
----- Forwarded message from "Stephen P. Wilson" <s.wilson at eris.qinetiq.com> -----
From: "Stephen P. Wilson" <s.wilson at eris.qinetiq.com>
Subject: Problems with distcc cross-compilation: Sharp Zaurus <-> Linux...
Date: Mon, 04 Nov 2002 18:12:57 +0000
To: mbp at sourcefrog.net
Cc: rik at kde.org, Stephen Wilson <s.wilson at eris.qinetiq.com>
Organization: QinetiQ
X-Mailer: Mozilla 4.79 [en] (WinNT; U)
Hi,
I'm trying to use distcc to get cross-compilation between a Sharp Zaurus
and my linux desktop machine (I'd like to run ./configure on the Zaurus
and have distcc pass everything across to the linux box for actual
compilation).
So, I grabbed a pre-compiled copy of distcc from
http://rikkus.info/zaurus.html#distcc, installed the package and
appropriate libraries, and the distcc client seems to run ok:
> # distcc
> distcc 0.7 armv4l-unknown-linux-gnu (protocol 1)
> built Aug 13 2002 04:22:47
I set the environment variables to point to my linux box (centipede):
> # export DISTCC_HOSTS="centipede localhost" CC=distcc CXX=dist-g++
(NB: I've tried this without the "localhost" bit (since the Zaurus
doesn't have enough space to install gcc locally) - but the results were
identical).
I create a (very simple) little test program to test with:
> # cat test.c
> #include<stdio.h>
>
> int main()
> {
> printf("hello world!\n");
>
> return(1);
> }
In the meantime, on the server:
I installed a copy of distcc too:
> !sharp-dev-armuser at centipede:~$ distcc
> distcc 0.7 i586-pc-linux-gnu (protocol 1)
(NB: Originally I had a copy of 0.12 - but thought the errors might be
due to differing versions - so downloaded the same version as I had
precompiled for the Zaurus)
The user account has some variables set:
> !sharp-dev-armuser at centipede:~$ cat ./dev-arm-qpe.sh
> CROSSCOMPILE=/opt/Embedix/tools
> QPEDIR=/opt/Qtopia/sharp
> QTDIR=/opt/Qtopia/sharp
> PATH=$QTDIR/bin:$QPEDIR/bin:$CROSSCOMPILE/bin:$PATH
> TMAKEPATH=/opt/Qtopia/tmake/lib/qws/linux-sharp-g++/
> LD_LIBRARY_PATH=$QTDIR/lib:/usr/X11R6/lib
> PS1='!sharp-dev-arm\u@\h:\w\$ '
> export CROSSCOMPILE QPEDIR QTDIR PATH LD_LIBRARY_PATH TMAKEPATH PS1
> echo "Altered environment for Sharp Zaurus SL-5000D Development ARM"
>
> !sharp-dev-armuser at centipede:~$ . ./dev-arm-qpe.sh
> Altered environment for Sharp Zaurus SL-5000D Development ARM
And, just to make sure that the program will compile here, I create
another copy and compile it locally:
> !sharp-dev-armuser at centipede:~$ cat test.c
> #include<stdio.h>
>
> int main()
> {
> printf("hello world!\n");
>
> return(1);
> }
>
> !sharp-dev-armuser at centipede:~$ gcc -o test test.c
> !sharp-dev-armuser at centipede:~$ ls -la test*
> -rwxrwxr-x 1 user user 11884 Nov 4 17:04 test*
> -rw-r--r-- 1 root root 76 Nov 4 16:06 test.c
> !sharp-dev-armuser at centipede:~$ ./test
> hello world!
Finally, I start distcc in daemon mode:
> !sharp-dev-armuser at centipede:~$ distccd --daemon
Which logs in /var/log/messages:
> Nov 4 17:00:24 centipede distccd[25677]: distccd (version 0.7, built Nov 4 2002 16:18:52) listening on port 4200
> Nov 4 17:00:24 centipede distccd[25678]: (dcc_parent_loop) waiting to accept connection
So, at this point - I have a single client that isn't running the daemon
(the Zaurus) and a single server running the daemon (centipede). So, on
the client, I try and run gcc through distcc:
> # distcc gcc -o test test.c
> distcc[30606] (dcc_scan_args) scanning arguments: gcc -o test test.c
> distcc[30606] (dcc_scan_args) found object file "test"
> distcc[30606] (dcc_scan_args) found input file "test.c"
> distcc[30606] (dcc_scan_args) compiler apparently called not for compile
> distcc[30606] (dcc_parse_hosts) found tcp token "centipede"
> distcc[30606] (dcc_try_lock_host) locked /tmp/distcc_00000000/lock_centipede_0000000
> distcc[30606] (dcc_pick_buildhost) building on centipede
> distcc[30606] exec on localhost: gcc -o test test.c
> distcc[30606] (dcc_spawn_child) forking to execute gcc -o test test.c
> distcc[30606] (dcc_spawn_child) child started as pid30607
> distcc[30607] ERROR: (dcc_execvp) failed to exec gcc: No such file or directory
> distcc[30607] (dcc_exit) exit: code 104; self: 0.000000 user 0.000000 sys; children: 0.000000 user 0.000000 sys
> distcc[30606] (dcc_collect_child) child 30607 terminated with status 0x6800
> distcc[30606] (dcc_report_rusage) gcc resource usage: 0.000000s user, 0.000000s system
> distcc[30606] ERROR: (dcc_critique_status) compile on localhost failed with exit code 104
> distcc[30606] (dcc_exit) exit: code 104; self: 0.000000 user 0.030000 sys; children: 0.000000 user 0.000000 sys
Interestingly, nothing appears to be sent to the server at this point -
I couldn't pick up any network traffic (using tcpdump/ethereal) or
anything in the logs. However, if I use a slightly different command
("distcc gcc -o test -c test.c" instead of "distcc gcc -o test test.c")
I get the following:
> # distcc gcc -o test -c test.c
> distcc[30615] (dcc_scan_args) scanning arguments: gcc -o test -c test.c
> distcc[30615] (dcc_scan_args) found object file "test"
> distcc[30615] (dcc_scan_args) found input file "test.c"
> distcc[30615] compile from test.c to test
> distcc[30615] (dcc_parse_hosts) found tcp token "centipede"
> distcc[30615] (dcc_try_lock_host) locked /tmp/distcc_00000000/lock_centipede_0000000
> distcc[30615] (dcc_pick_buildhost) building on centipede
> distcc[30615] (dcc_set_output) changed output from "test" to "/tmp/distcc_00000000/cppout_0030615.i"
> distcc[30615] (dcc_set_output) command after: gcc -o /tmp/distcc_00000000/cppout_0030615.i -E test.c
> distcc[30615] (dcc_spawn_child) forking to execute gcc -o /tmp/distcc_00000000/cppout_0030615.i -E test.c
> distcc[30615] (dcc_spawn_child) child started as pid30616
> distcc[30616] ERROR: (dcc_execvp) failed to exec gcc: No such file or directory
> distcc[30615] exec on centipede: gcc -o test -c test.c
> distcc[30616] (dcc_exit) exit: code 104; self: 0.000000 user 0.010000 sys; children: 0.000000 user 0.000000 sys
> distcc[30615] (dcc_open_socket_out) client got connection to centipede port 4200 on fd4
> distcc[30615] (dcc_collect_child) child 30616 terminated with status 0x6800
> distcc[30615] (dcc_report_rusage) cpp resource usage: 0.000000s user, 0.010000s system
> distcc[30615] ERROR: (dcc_critique_status) cpp on centipede failed with exit code 104
> distcc[30615] (dcc_exit) exit: code -1; self: 0.020000 user 0.010000 sys; children: 0.000000 user 0.010000 sys
And the following is logged on the server (slightly sanitized to hide
name/IP):
> Nov 4 17:02:53 centipede distccd[25483]: (dcc_serve_connection) up to 2 children
> Nov 4 17:02:53 centipede distccd[25483]: (dcc_reap_kids) down to 1 kids
> Nov 4 17:02:53 centipede distccd[25483]: (dcc_reap_kids) child 25554 exited: status 255
> Nov 4 17:02:53 centipede distccd[25483]: (dcc_parent_loop) waiting to accept connection
> Nov 4 17:02:53 centipede distccd[25637]: (dcc_log_clientname) connection from SANITIZED (xxx.xxx.xxx.xxx)
> Nov 4 17:02:53 centipede distccd[25637]: (dcc_accept_job) got arguments: gcc -o test -c test.c
> Nov 4 17:02:53 centipede distccd[25637]: compile from test.c to test
> Nov 4 17:02:53 centipede distccd[25637]: input file test.c, output file test
> Nov 4 17:02:53 centipede distccd[25637]: CRITICAL! (dcc_readx) unexpected eof on fd5
> Nov 4 17:02:53 centipede distccd[25637]: ERROR: (dcc_expect_token) read failed while waiting for token "DOTI"
> Nov 4 17:02:53 centipede distccd[25637]: (dcc_exit) exit: code -1; self: 0.010000 user 0.010000 sys; children: 0.000000 user 0.000000 sys
I did notice that there were a number of files stored in the
/tmp/distcc_* directory on the server:
> [root at centipede distcc_000001f5]# pwd
> /tmp/distcc_000001f5
> [root at centipede distcc_000001f5]# ls -l
> total 32
> -rw-rw-r-- 1 user user 713 Nov 4 16:21 server_0025404.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:28 server_0025520.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:30 server_0025524.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:31 server_0025528.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:33 server_0025532.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:51 server_0025547.out
> -rw-rw-r-- 1 user user 713 Nov 4 16:53 server_0025554.out
> -rw-rw-r-- 1 user user 713 Nov 4 17:02 server_0025637.out
I tried executing these both locally to the server, and on the Zaurus -
however, in both cases the files were not executable (obviously, I
chmod'd them to u+x first).
Just in case it's of any use - this is what the contents of the files
looks like:
> [root at centipede distcc_000001f5]# od -c server_0025404.out
> 0000000 177 E L F 001 001 001 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000020 001 \0 003 \0 001 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000040 À \0 \0 \0 \0 \0 \0 \0 4 \0 \0 \0 \0 \0 ( \0
> 0000060 \t \0 006 \0 \b \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0
> 0000100 0 1 . 0 1 \0 \0 \0 \0 G C C : ( G
> 0000120 N U ) 2 . 9 6 2 0 0 0 0 7 3
> 0000140 1 ( M a n d r a k e L i n u
> 0000160 x 9 . 0 2 . 9 6 - 0 . 8 0 m
> 0000200 d k ) \0 \0 . s y m t a b \0 . s t
> 0000220 r t a b \0 . s h s t r t a b \0 .
> 0000240 t e x t \0 . d a t a \0 . b s s \0
> 0000260 . n o t e \0 . c o m m e n t \0 \0
> 0000300 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> *
> 0000340 \0 \0 \0 \0 \0 \0 \0 \0 033 \0 \0 \0 001 \0 \0 \0
> 0000360 006 \0 \0 \0 \0 \0 \0 \0 4 \0 \0 \0 \0 \0 \0 \0
> 0000400 \0 \0 \0 \0 \0 \0 \0 \0 004 \0 \0 \0 \0 \0 \0 \0
> 0000420 ! \0 \0 \0 001 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0
> 0000440 4 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000460 004 \0 \0 \0 \0 \0 \0 \0 ' \0 \0 \0 \b \0 \0 \0
> 0000500 003 \0 \0 \0 \0 \0 \0 \0 4 \0 \0 \0 \0 \0 \0 \0
> 0000520 \0 \0 \0 \0 \0 \0 \0 \0 004 \0 \0 \0 \0 \0 \0 \0
> 0000540 , \0 \0 \0 \a \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000560 4 \0 \0 \0 024 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000600 001 \0 \0 \0 \0 \0 \0 \0 2 \0 \0 \0 001 \0 \0 \0
> 0000620 \0 \0 \0 \0 \0 \0 \0 \0 H \0 \0 \0 < \0 \0 \0
> 0000640 \0 \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0
> 0000660 021 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000700 204 \0 \0 \0 ; \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0000720 001 \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 002 \0 \0 \0
> 0000740 \0 \0 \0 \0 \0 \0 \0 \0 ( 002 \0 \0 200 \0 \0 \0
> 0000760 \b \0 \0 \0 \b \0 \0 \0 004 \0 \0 \0 020 \0 \0 \0
> 0001000 \t \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001020 ¨ 002 \0 \0 ! \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001040 001 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001060 \0 \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0
> 0001100 \0 \0 \0 \0 004 \0 ñ ÿ \0 \0 \0 \0 \0 \0 \0 \0
> 0001120 \0 \0 \0 \0 003 \0 001 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001140 \0 \0 \0 \0 003 \0 002 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001160 \0 \0 \0 \0 003 \0 003 \0 022 \0 \0 \0 \0 \0 \0 \0
> 0001200 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001220 \0 \0 \0 \0 003 \0 004 \0 \0 \0 \0 \0 \0 \0 \0 \0
> 0001240 \0 \0 \0 \0 003 \0 005 \0 \0 s e r v e r _
> 0001260 0 0 2 5 4 0 4 . i \0 g c c 2 _ c
> 0001300 o m p i l e d . \0
> 0001311
And now, I guess I'm a little out of my depth. I did start looking at
the distcc source code - but it's a while since I've done any
programming and I just ended up getting even more confused than I am
already. ;-)
Any idea what might be going wrong, or how I might fix it?
Cheers,
Steve.
--
Stephen Wilson
Senior Security Consultant
Security Health Check
L305/9, QinetiQ, St Andrews Rd, Malvern, Worcs, WR14 3PS
Tel: 01684 894153 Fax: 01684 897417
----- End forwarded message -----
--
Martin
More information about the distcc
mailing list