rsync crashing Redhat RHEL3.0 server
paulh at hamjudo.com
Tue Aug 31 13:42:11 GMT 2004
On Tue, 31 Aug 2004, Paul Slootman wrote:
> Sounds like you're running out of memory (and swap as well).
> I'd suggest adding memory (adding more swap doesn't make it faster...)
Adding swap space lets you figure out how much memory to buy, or ...
> or rsyncing smaller bits at a time if possible
how much more you need to subdivide backups ...
> (if there are many
> hardlinks across directory trees it may not be feasible to do parts
> separately as those hardlinks will be lost, for example).
if possible. If none of those options work, you can use ulimit -v to
limit the process size of the rsync jobs, so they die before damaging the
rest of the system.
To add swap, I use
dd if=/dev/zero of=/fullpathhere/swap1 bs=1M count=1024
repeat for swap2, swap3, etc... The add them all to /etc/fstab.
The command "top" shows how much memory and swap is in use. Top has a
batch mode, so you can do things like tell it to check once a minute for
24 hours with a command line like:
top -b -d 60 -n 1440 > /fullpathhere/top.`date +%F`.log
Put that in a cron job that fires off at midnight. After something goes
wrong, look at the logfile and see what was going on. Maybe there is
another process that sometimes uses lots of memory.
Notes, add another cron job to clean up the top logfiles. See "man top"
for all the spiffy options to get it to log more of what you want to see.
More information about the rsync