rsync 2.6.0 - suspected memory leak bug
kelly.garrett at level3.com
Wed Jan 21 15:35:37 GMT 2004
Wayne Davison <wayned <at> samba.org> writes:
> On Tue, Jan 20, 2004 at 06:16:34PM -0800, jw schultz wrote:
> > As far as i can tell Under inetd each connection should get
> > independant rsync process(es) which all exit so there would be no
> > rsync processes running unless there is an active connection.
> Yes, that's my understanding as well. He also said that the receiving
> side is initiating the connection (and having the problem), so I'm
> assuming that he's not complaining about the daemon side.
> One other thing I thought of: Kelly, check for lingering rsync
> processe(s) in the process table. That's the only way I know that
> memory could continue to be allocated at the end of a transfer: if
> one of the processes didn't actually exit.
There are no residual rsync processes running after the transfer. I think you
hit it on the head when you indicated that Linux would use available mem for
cacheing that is messing up. Rsync seems to be exiting correctly, and I have
tried it with several different versions of rsync. The way that the kernel is
currently compiled/configured seems not to be releasing the diskcache memory
back to other processes when needed, and the machine crashes once all available
RAM and swap has been consumed by the cache during the rsync transfer. So, I
do not beleive it is rsync's "fault" -- rsync merely exposed a flaw in the OS
due to the massive size of the transfers we are doing with it. My appologies
to the rsync team, it is an excellent tool and had I not been impressed with it
I would not have bothered tracking down the problem.
Does anyone know how to build a version of the kernel that either does no disk
cacheing (we have very fast RAID processors and SCSI disks on the machine) or
limit the amount of cache that the system will allocate for disk?
Any help would be appreciated.
More information about the rsync