[distcc] Re: Distcc freezes during compile

[Dag Wieers]

On Tue, 3 Feb 2004, Martin Pool wrote:

> On  2 Feb 2004, storri at torri.org wrote:
> > Distcc version 2.12.1
> > Platform: Redhat 9
> > Compiler: gcc (GCC) 3.2.2 20030222 (Red Hat Linux 3.2.2-5)
> > 
> > Action: Attempting to compile personal application
> 
> > There are six distcc hosts being used for the compiler process. Two
> > hosts are running stock RedHat 9 kernels. Four hosts running distccd
> > and the controller using distcc are using a custom 2.4.18 kernel
> > with preemption. During the compilation process all works normally
> > until the contoller freezes. The machine locked solid with no
> > response to Ctrl-C (accessing controller via ssh) or direct keyboard
> > access. The only method to recover control is to soft reboot the
> > machine via the front switch. The machines were able to compile work
> > with the 2.5 version of distcc. Attached is a compressed tarball of
> > the build process from the controller.
> 
> This is a kernel bug -- applications should never hang the machine.
> Switch to stock 2.4.24 and see if the problem can still be reproduced.
> If it can, let us know.  If that works, see if the 2.4.24 preemption
> patch works.

There has been a kernel memory leak in the 2.4.18/2.4.20 RH kernels. 
Depending on the use your machine could run out of memory in a couple of 
hours up to weeks. I've seen it happen on boxes that have high IO-load and 
CPU-usage, mainly with rrdtool and mrtg. You can find more information 
here:

	Red Hat Bugzilla #97546
	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=97546

	[rhl-devel-list] Kernel eating memory, ends up trashing
    	http://www.redhat.com/archives/rhl-devel-list/2003-September/msg00032.html

	A nice graph
	http://dag.wieers.com/attic/rmon-breeg-mem-3months-800x120.png

It may have been fixed in recent RH80/RH9 kernels.

--   dag wieers,  dag at wieers.com,  http://dag.wieers.com/   --
[Any errors in spelling, tact or fact are transmission errors]