[distcc] garbage transferred instead of preprocessed source

[Zdenek Behan]

Hi,

I encountered a very strange problem with distcc. Let me explain:

I have 2 machines (both gentoo). One is i686 (fast) and the other is ppc 
(slow). I have a working(tested) ppc crosscompiler on i686 and native 
compiler on ppp [same versions - 3.4.6]

I emerged exactly the same version of distcc on both (tried multiple 
versions), and ran them with:

(slow machine)
PATH="/usr/powerpc-unknown-linux-gnu/bin:/usr/powerpc-unknown-linux-gnu/gcc-bin/3.4.6/" 
/usr/bin/distccd -p 55555 -N 10 --allow 192.168.1.0/24 
--listen=192.168.1.15 --no-detach --user distcc --log-stderr

(fast machine)
PATH="/usr/powerpc-unknown-linux-gnu/bin:/usr/powerpc-unknown-linux-gnu/gcc-bin/3.4.6/" 
/usr/bin/distccd -p 55555 -N 10 --allow 192.168.1.0/24 
--listen=192.168.1.201 --no-detach --user distcc --log-stderr


I put both hosts (201, 15) into /etc/distcc/hosts.

Daemons work fine until i try to actually compile something.

--
#include <stdio.h>

int main( int argc, char ** argv )
{
         printf("Hello world!\n");

         return 1;
}
--

I created simple hello.c to demonstrate. The command used is:
distcc powerpc-unknown-linux-gnu-gcc -c -o hello.o hello.c

Now i have 4 variants of using distcc. Fast to Fast (localhost) Fast to 
Slow, Slow to Fast and Slow to Slow.

When doing any of the pointless variants (F->S, F->F, S->S), distccd 
creates /tmp/distccd_key.i on the local machine containing preprocessed 
source and within fraction of a second, it's done. Verbose distccd 
output says something like:

distccd[12392] (dcc_check_client) connection from 192.168.1.15:1895
distccd[12392] compile from hello.c to hello.o
distccd[12392] (dcc_r_file_timed) 16695 bytes received in 0.001372s, 
rate 11883kB/s
distccd[12392] (dcc_collect_child) cc times: user 0.170000s, system 
0.040000s, 501 minflt, 1030 majflt
distccd[12392] powerpc-unknown-linux-gnu-gcc hello.c on localhost 
completed ok
distccd[12392] job complete



In the last variant (Slow -> Fast), it creates /tmp/distccd_key.i as 
well, however, what it contains can hardly be compared to preprocessed 
source. It's basically a binary file containing a random dump of some 
disk data. I have a copy of such a file in case anyone wants to see it, 
but there's not much to see, really.

Naturally this fails with megabytes long error log going as following
/tmp/distccd_237f6a6d.i:122: error: stray '\242' in program
/tmp/distccd_237f6a6d.i:122: error: stray '\160' in program
/tmp/distccd_237f6a6d.i:122: error: stray '\195' in program
/tmp/distccd_237f6a6d.i:122: error: stray '\242' in program
...

Output looks like this:
distccd[11797] (dcc_check_client) connection from 192.168.1.15:1894
distccd[11797] compile from hello.c to hello.o
distccd[11797] (dcc_r_file_timed) 16695 bytes received in 0.002057s, 
rate 7926kB/s
distccd[11797] (dcc_collect_child) cc times: user 0.425935s, system 
0.989849s, 889 minflt, 0 majflt
distccd[11797] powerpc-unknown-linux-gnu-gcc hello.c on localhost failed
distccd[11797] job complete

Notice the size file size actually being the same. It's the content that 
is scrambled, for reason completely unknown to me. Neither side does 
crash, only report the huge error log and then go on.

Just for the record, distcc is built on both systems natively with 
native compiler (same version - 3.4.6), glibc versions are not the same, 
but i can hardly imagine that being a problem.


Can anyone help me, or at least point me to where i should be looking 
for the problem? This seems to be purely distcc issue, as it never gets 
to actually compiling anything, besides, i believe my crosscompiler 
setup is correct.

My first guess was endianity swap (ppc is big endian), but since there 
is some totally out of place text mixed up with garbage binary data in 
the temporary file, i think that's not the solution. So now i'm left 
with being completely clueless, and any help will be appreciated.


Zdenek