rsync 2.6.0 - suspected memory leak bug

[Kelly Garrett]

Wayne Davison <wayned <at> samba.org> writes:

> 
> On Tue, Jan 20, 2004 at 06:16:34PM -0800, jw schultz wrote:
> > As far as i can tell Under inetd each connection should get
> > independant rsync process(es) which all exit so there would be no
> > rsync processes running unless there is an active connection.
> 
> Yes, that's my understanding as well.  He also said that the receiving
> side is initiating the connection (and having the problem), so I'm
> assuming that he's not complaining about the daemon side.
> 
> One other thing I thought of:  Kelly, check for lingering rsync
> processe(s) in the process table.  That's the only way I know that
> memory could continue to be allocated at the end of a transfer:  if
> one of the processes didn't actually exit.
> 
> ..wayne..

There are no residual rsync processes running after the transfer.  I think you 
hit it on the head when you indicated that Linux would use available mem for 
cacheing that is messing up.  Rsync seems to be exiting correctly, and I have 
tried it with several different versions of rsync.  The way that the kernel is 
currently compiled/configured seems not to be releasing the diskcache memory 
back to other processes when needed, and the machine crashes once all available 
RAM and swap has been consumed by the cache during the rsync transfer.  So, I 
do not beleive it is rsync's "fault" -- rsync merely exposed a flaw in the OS 
due to the massive size of the transfers we are doing with it.  My appologies 
to the rsync team, it is an excellent tool and had I not been impressed with it 
I would not have bothered tracking down the problem.

Does anyone know how to build a version of the kernel that either does no disk 
cacheing (we have very fast RAID processors and SCSI disks on the machine) or 
limit the amount of cache that the system will allocate for disk?

Any help would be appreciated.

Kelly