[distcc] Re: distributing preprocsessing

[Christian Gerlach]

Martin Pool wrote:

> On Sat 2003-09-13, Christian Gerlach wrote:
> 
>> Hello,
>> 
>> I am working for a company which is currently using teambuilder for
>> distributed compile jobs. We have lots of really fast machines and a
>> common NFS-share for our homes.
>> I have now tried using distcc in combination with ccache and I am really
>> impressed. The problem which I see (teambuilder has this too) is that the
>> preprocessing is only done by one machine.
>> My question is:
>> Is there any chance to distribute the preprocsessing step?
>> Using:
>> - The same system/compiler
>> - NFS share
>> Is this possible and do you see any benefits?
>> Maybe I am completely wrong.
> 
> Hi Christian,
> 
> Yes, you can certainly do this.  I know some people have written
> programs to do similar things, but I can't find one on the web at the
> moment.  Please let us know if you find one.
> 
> If you can rely on all the machines seeing the same filesystem then
> things are much easier obviously.  distcc just provides a way to cope
> without it.
> 
> distcc can't do it, and I don't really know if it would be in scope to
> add it.  There would not be much in common aside from at the broadest
> level.  Most of the code in distcc is concerned with deciding which
> jobs can be distributed, and sending them across the network, and
> neither of them would be needed.
> 

But adding this functionality to distcc seems to me not a big deal.
The only things you have to do:
- Do not distribute the prepocessed file, but the working directory and the
compiler command line.
- Patch distccd to start the compiler in the given directory with the
options. Or even better: start ccache with the command line, using a cache
in a shared nfs-directory.

Is there anything I have missed?