Common memory usage questions

[Matt McCutchen]

On Tue, 2010-02-09 at 21:46 -0500, W. Michael Petullo wrote:
> I have started investigating the memory usage of the rsync algorithm since
> installing OpenWRT on a Linksys WRT160NL router with only 32MB of RAM.

Wow, that's pretty tight.

> Rsync consumes all of the memory on my router even when I ensure
> incremental scanning is used. This is not what I expect. The directory I
> am recursively synchronizing contains 56,545 directories in its tree.
> However, the maximum number of entries in any one directory is only 2,783.

See:

http://lists.samba.org/archive/rsync/2007-August/018193.html

The number of files that rsync tries to maintain in the active file
lists at one time is controlled by {MIN,MAX}_FILECNT_LOOKAHEAD in
rsync.h.  I did some tests with "ulimit -v", and reducing those values
helped significantly.  That still may not be enough to fit rsync into 32
MB.

-- 
Matt