rsync memory usage

[Lutz Pressler]

Hi,

we are using rsync to mirror large trees (>> 50 GB, >> 2 mio files)
offsite (rsync -e ssh with forced ssh command on the remote side).
The main problem occuring is memory usage (especially as the remote
system has no swap space configured for security reasons):

It seems that the rsync processes' RSS on both sides increases with
the number of files in the tree. Rsyncing the whole tree at once is
not possible anymore, as the remote system gets out of memory.

At the moment, we're rsyncing (sub)subdirectories seperately to not
reach the memory limit. But this is not really feasible anymore,
because of dynamic spread of content and limitations by the ssh
forced command method.

Why is rsync holding data for the whole tree in memory?

Any idea for a solution to this problem?

Thank you,
  Lutz Pressler

P.S.: rsync versions 2.5.5 on the remote and 2.4.6 on the local side.
      Ask for exact command lines if necessary.


-- 
  _              |  Lutz Pressler          |  Tel: ++49-551-3700002
 |_     |\ |     |  Service Network GmbH   |  FAX: ++49-551-3700009
 ._|ER  | \|ET   |  Bahnhofsallee 1b       |   mailto:lp at SerNet.DE
Service Network  |  D-37081 Goettingen     |  http://www.SerNet.DE/