rsync crashing Redhat RHEL3.0 server

[Paul Haas]

On Tue, 31 Aug 2004, Paul Slootman wrote:

> Sounds like you're running out of memory (and swap as well).
> I'd suggest adding memory (adding more swap doesn't make it faster...)

Adding swap space lets you figure out how much memory to buy, or ...

> or rsyncing smaller bits at a time if possible

how much more you need to subdivide backups ...

> (if there are many
> hardlinks across directory trees it may not be feasible to do parts
> separately as those hardlinks will be lost, for example).

if possible.  If none of those options work, you can use ulimit -v to 
limit the process size of the rsync jobs, so they die before damaging the 
rest of the system.


To add swap, I use
   mkdir /fullpathhere
   dd if=/dev/zero of=/fullpathhere/swap1 bs=1M count=1024
   swapon /fullpathhere/swap1
repeat for swap2, swap3, etc...  The add them all to /etc/fstab.

The command "top" shows how much memory and swap is in use.  Top has a 
batch mode, so you can do things like tell it to check once a minute for 
24 hours with a command line like:

  top -b -d 60 -n 1440 > /fullpathhere/top.`date +%F`.log

Put that in a cron job that fires off at midnight.  After something goes 
wrong, look at the logfile and see what was going on.  Maybe there is 
another process that sometimes uses lots of memory.

Notes, add another cron job to clean up the top logfiles. See "man top" 
for all the spiffy options to get it to log more of what you want to see.