memory requirements was RE: out of memory in build_hash_table

Eric Whiting ewhiting at amis.com
Wed Apr 24 07:28:01 EST 2002


"Granzow, Doug (NCI)" wrote:
> 
> From what I've observed by running top while rsync is running, its memory
> usage appears to grow gradually, not exponentially.  A rsync may take
> several hours to complete.  (I have one running now that started over four
> hours ago.  The filesystem contains 236 GB of data in 2.4 million files.  It
> is currently taking up 1351MB of memory on the mirror server and 646M on the
> source server.)  All filesystems are veritas filesystem, in case that is
> relevant.

I just killed a 750M rsync (2.5.5) process. It had been runing for over
12 hours. (around 62G in 1.2 million files)..  This was a local rsync
without any ssh/remsh/rsyncd involved. I went back to the 'for file in
*;do rsync blaa;done' method. This helps it run in a reasonable amount
of time and RAM. I'd still like to be able to run a rsync on the
toplevel (without the for loop...)

eric




More information about the rsync mailing list