[distcc] compiling with distcc on x86_64 from within a 32-bit chroot?

[Audio Phile]

OK.  I will scrap the idea of not using the chroot.  With your script and symlinks setup, and with my chroot on the file system, I gave this a try:

$ make -j8 ARCH=x86 CROSS_COMPILE=i686-pc-linux-gnu-
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
  HOSTCC  scripts/kallsyms
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  HOSTCC  scripts/pnmtologo
  HOSTCC  scripts/conmakehash
  HOSTCC  scripts/bin2c
  CC      arch/x86/kernel/asm-offsets.s
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/mod/sumversion.o
fixdep: error opening config file: /usr/lib/gcc/i686-pc-linux-gnu/4.6.0/include/stdarg.h: No such file or directory
make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 2
make: *** [prepare0] Error 2
make: *** Waiting for unfinished jobs....
  HOSTLD  scripts/mod/modpost

Also, appending the CC=i686-pc-linux-gnu gave a similar result

$ make -j8 prepare ARCH=x86 CROSS_COMPILE=i686-pc-linux-gnu-gcc- CC=i686-pc-linux-gnu-gcc
  CHK     include/linux/version.h
  HOSTCC  scripts/basic/fixdep
  CHK     include/generated/utsrelease.h
  HOSTCC  scripts/basic/docproc
  CC      kernel/bounds.s
  GEN     include/generated/bounds.h
  CC      arch/x86/kernel/asm-offsets.s
fixdep: error opening config file: /usr/lib/gcc/i686-pc-linux-gnu/4.6.0/include/stdarg.h: No such file or directory
make[1]: *** [arch/x86/kernel/asm-offsets.s] Error 2
make: *** [prepare0] Error 2

I'm beginning to this this is over my head :)

--- On Mon, 6/27/11, Benjamin R. Haskell <distcc at benizi.com> wrote:

> From: Benjamin R. Haskell <distcc at benizi.com>
> Subject: Re: [distcc] compiling with distcc on x86_64 from within a 32-bit chroot?
> To: "Audio Phile" <da_audiophile at yahoo.com>
> Cc: "Martin Pool" <mbp at sourcefrog.net>, distcc at lists.samba.org
> Date: Monday, June 27, 2011, 6:52 PM
> On Mon, 27 Jun 2011, Audio Phile
> wrote:
> 
> >> Too many double-quotes there?  It should end
> with
> >> (quotes included): $0 "$@"
> > 
> > That was a typeo.  The real script is correct.
> 
> Okay.
> 
> 
> > If I understand everything you wrote, I only need a
> 32-bit chroot on one of my boxes (the daemon)?  All the
> server PCs in my distcc cluster can have their native x86_64
> gcc's?
> > 
> > Am I right?
> 
> I don't think so, but it depends.  Without considering
> SSH-based or other solutions, each PC in a distcc cluster on
> which you want to run compilation jobs has to have the
> ability to target the platform for which you're trying to
> compile.
> 
> It might be possible to set it up such that even
> configuration-related compilation takes place remotely (or
> there might even be very little of that in a kernel
> build).  The only example where I could see this being
> beneficial is if you have a set of underpowered workstations
> and want to offload all compilation to a vastly bigger
> central box.
> 
> Otherwise, if you want to spread the compilation around to
> all computers in the cluster, you should run a distcc daemon
> on each.  And every PC running the daemon needs to have
> the chroot environment.
> 
> 
> > Was was intrigued by your statement about cross
> compiling with make 3.82 and had a look in the linux
> kernel's Makefile and I found the following:
> > 
> > # Cross compiling and selecting different set of
> gcc/bin-utils
> > #
> ---------------------------------------------------------------------------
> > #
> > # When performing cross compilation for other
> architectures ARCH shall be set
> > # to the target architecture. (See arch/* for the
> possibilities).
> > # ARCH can be set during invocation of make:
> > # make ARCH=ia64
> > # Another way is to have ARCH set in the environment.
> > # The default ARCH is the host where make is
> executed.
> > 
> > # CROSS_COMPILE specify the prefix used for all
> executables used
> > # during compilation. Only gcc and related bin-utils
> executables
> > # are prefixed with $(CROSS_COMPILE).
> > # CROSS_COMPILE can be set on the command line
> > # make CROSS_COMPILE=ia64-linux-
> > # Alternatively CROSS_COMPILE can be set in the
> environment.
> > # A third alternative is to store a setting in .config
> so that plain
> > # "make" in the configured kernel build directory
> always uses that.
> > # Default value for CROSS_COMPILE is not to prefix
> executables
> > # Note: Some architectures assign CROSS_COMPILE in
> their arch/*/Makefile
> > 
> > So as I read that, I can use your script and symlinks
> to my chroot root on my native x86_64 environment as you
> showed me with the hello.c example by doing:
> > 
> > $ export ARCH=x86
> > $ make -j8 bzImage CC=i686-pc-linux-gnu-gcc
> > 
> > Do I have this right?  I posted the entire
> Makefile to pastebin if you need to see more in it.
> 
> From the help text in those comments, it sounds like you
> actually want:
> 
> $ make -j8 ARCH=x86 CROSS_COMPILE=i686-pc-linux-gnu-
> bzImage
> 
> The discussion of 'prefixed executables' is why we
> previously set up all those symlinks.
> 
> -- Best,
> Ben