Server crashed using rsync

[Paul Haas]

On Wed, 28 Jan 2004, Eugen Luca wrote:

> I'm trying to make a backup using this command

>     rsync -auvH /home/ /bak --delete --bwlimit=1000 --status

> server load has been increased so much and the server crashed, as well
> has gone out of memory
> My Server is a Dual Xeon 2.0 GHz with 2GB of Memory + 1GB Swap.

In addition to using the new improved CVS version of rsync as others have
suggested, you should also make it so your system won't die just because
there is a large process.  If you add some more swapspace, the system will
slow down when you run big programes.  This is painfull, but far less
painfull than whatever happens when it runs out of memory.

As long as you've got disk space you can add more swap easily on almost
any modern OS.  On Linux, the commands are:

  dd if=/dev/zero of=swapfile bs=1M count=2K
  swapon swapfile

You can add the swap area to /etc/fstab so it will be used on every
reboot.  It is much easier to debug a slow system than to debug a crashed
system.  If the swapfile is on a slow disk, you can make it low priority,
so it is used last.  Many Linux versions limit you to 2Gbytes per
swapfile, but you can use several at the same time.

You should use rsync's -exclude option so you don't back up the swapfile.

I haven't run a system out of memory in a while.  When I've done it in the
past, bad things happened, but the system didn't crash, I just wished it
had.

> Could be that there are too many files, about 5.000.000, to be backed up
> ?
> The way the files are structured make very difficult to create several
> backup with less files.
> Is there any one else facing issues like this ? Is there any workaround
> on this ?
>
>
> Thank you,
> Eugen Luca
>
--
Paul Haas
paulh at Hamjudo.com