[distcc] Higher compression

[Matias D'Ambrosio]

On Tuesday 25 September 2007 21:17:00 you wrote:
> On Tue, 2007-09-25 at 09:53 -0300, Matias D'Ambrosio wrote:
> > I have been using distcc for a couple of days with LZO compression,
> > but I have found it insufficient due to the very low bandwidth
> > available. The main system is a laptop with a serial port (10KB/sec)
> > connection to my main desktop which gives it access to the LAN, the
> > volunteer is connected through ethernet to the desktop, but obviously
> > the limit is the serial cable.
[snip]
> >  I have been doing some quick research and it appears 7z is the best
> > format (that is easily available, at least) with bzip2 a close second
> > (though not as good with object files). There is a 7z SDK with C
> > source and I might try to make that work with distcc, I have only
> > looked superficially at distcc's source so I don't know how easy this
> > would be, but I was curious to know if anybody is interested/working
> > in the same problem.
>
> That would certainly be interesting to try.  I'm not sure off hand but I
> think there is a field indicating what kind of compression is in use,
> and that could be extended to support 7z.
>
> It's not so much compression of the object files that matters as
> compression of the preprocessed source, which is highly compressible and
> much larger.
>
> You might check first just turning up the lzo compression level.
>
 I went over the 7zip SDK and it didn't have a C compressor, just a 
decompressor, and I generally didn't like it. So I went with bzip2 instead.
 I basically followed the LZO code, most of the new code is in 
compress_bz2.c . I should mention that currently it links to the system 
libbz2, is that a problem? I guess I could #ifdef all the code, but I don't 
know much about configure scripts, I just managed to get it to link libbz2 
(and not the proper way, I'm sure).
 Currently, the code compiles. I'm hoping to test it soonish and evaluate 
performance.