[distcc] Higher compression

[Martin Pool]

On Tue, 2007-09-25 at 09:53 -0300, Matias D'Ambrosio wrote: 
> I have been using distcc for a couple of days with LZO compression,
> but I have found it insufficient due to the very low bandwidth
> available. The main system is a laptop with a serial port (10KB/sec)
> connection to my main desktop which gives it access to the LAN, the
> volunteer is connected through ethernet to the desktop, but obviously
> the limit is the serial cable.
>  Although the network speed is the current bottleneck, distcc is the
> best option I have seen so far, since the laptop only has 32MB of RAM
> and hence can't compile some files (OOM crashes). Compounded to that
> is the fact that I use different linux distributions on my desktop and
> laptop, and thus I have so far been using a virtual machine in my
> desktop to compile, and then transferring the files to the laptop,
> which is cumbersome to say the least.
>  I have been doing some quick research and it appears 7z is the best
> format (that is easily available, at least) with bzip2 a close second
> (though not as good with object files). There is a 7z SDK with C
> source and I might try to make that work with distcc, I have only
> looked superficially at distcc's source so I don't know how easy this
> would be, but I was curious to know if anybody is interested/working
> in the same problem.
>  I should also mention that the laptop is a celeron coppermine 500MHz,
> so it has the processing power for 7z, and the volunteer is a duron
> 1400 with 256MB of RAM.

That would certainly be interesting to try.  I'm not sure off hand but I
think there is a field indicating what kind of compression is in use,
and that could be extended to support 7z.

It's not so much compression of the object files that matters as
compression of the preprocessed source, which is highly compressible and
much larger.

You might check first just turning up the lzo compression level.

Martin