Large RAM (> 4G) and rsync still dies?
rsync at med-web.com
rsync at med-web.com
Thu Nov 20 17:24:27 EST 2003
Hello. Hopefully someone can shed some light on this:
We've got a production server with _LOTS_ of files on it. The system is a dual
XEON with 4GB of RAM. During the evening, the load is very low. Linux shows
(via 'top') that approximately 3G of RAM is cached RAM, and 500M is buffered.
That, if my understanding is correct, should leave us with 3.5G of available
memory to draw upon.
Running rsync with --progress, (v2.5.6 I believe - just downloaded it a few
days ago) shows us that we reach a little over 8 million files before the
server starts telling us its killing processes because its out of RAM.
On the FAQ, it says that rsync should consume approximately 100bytes of memory
per file, on average. So, 8 million x 100 = 800M of RAM.
Why are we running out of RAM? Is there a way to tell the kernel not to use so
much memory for cache and buffers, and to leave more free? Is the kernel not
releasing the cache/buffer memory quick enough for rsync? I don't know,
otherwise I wouldn't be here asking these questions. =)
Thanks and hopefully someone out there has an answer!
Jason
More information about the rsync
mailing list