[distcc] Miscompilation with -march=native

[Maks Verver]

Hi everyone,

I've set up distcc to be able to compile packages for a host running Gentoo on 
a somewhat slow machine. Gentoo has a nice feature where you can tell the 
package manager to use distcc for all packages, automatically adjustting 
compiler paths and whatnot. In general, this works fine. There are few 
packages that call the compiler incorrectly (e.g. "gcc" instead 
of "i386-pc-linux-gnu") but since I've set-up distcc so that on the build 
nodes the default compiler is not in the path, if this occurs the build fails 
and I can restart it without using distcc. So far so good.

However, I do have problems with packages that call gcc with the -march=native 
option. This option causes gcc to generate code for the native platform and 
CPU (similar to specifying a specific architecture like 
e.g. -march=athlon-xp). The problem is that this causes the build nodes to 
generate instructions that are not available on the machine that initiated 
the build!

This is annoying because this miscompilation is not detected while compiling 
and linking (because the object files produced have a perfectly valid 
format). It's also not immediately apparent when running the executable, 
because not all object files are compiled remotely and even if they are, not 
all object files contain illegal instructions. The result is that an 
application may work fine for some inputs but may mysteriously abort with 
an "invalid instruction" exception when it happens to come across code in a 
miscompiled object file.

So my question with this is: is this a known problem? Is there a way to work 
around this, short of "don't use distcc for these packages"? In my opinion, 
distcc should either process requests locally when the -march=native option 
is given (just like operations like linking are always done locally), or 
(preferably?) rewrite them to read -march=arch-of-the-build-machine so the 
target architecture is the same on all build nodes, but I don't know if this 
is hard to implement (it's a very gcc-specific feature). Comments/opinions?

Kind regards,
Maks Verver.