script to rysnc to spare hard drive
Martin Schwenke
martin at meltin.net
Tue Mar 5 09:54:16 EST 2002
>>>>> "Gary" == Gary Peltola <clist at www-hosting.net> writes:
Gary> Bascially what i am looking at doing is mirroring the first
Gary> drive to the 2nd drive, and once that intial one is moved
Gary> over, have a cron run nightly (or when specified) that will
Gary> update any modified files from the cron run. I am looking to
Gary> have this as a perl script (or whatever) so i can easily
Gary> port it to all my servers across my network. So that if hard
Gary> drive 1 fails, i will just need to pop in hard drive two and
Gary> be up and running within minutes rather then hours (my last
Gary> outage was pretty major on my /boot partion and if had rysnc
Gary> setup in the manor i am speaking of, downtime would have
Gary> went from 8 hours to less then 30 minutes....)
Gary> Does anyone have such examples of what i can do to
Gary> accomplish this?
Below is a couple of scripts that do this... but wait there's still
more. :-)
The first script does this:
* Backup filesystems on /dev/hda to /dev/hdb, which gets mounted under
/backup.
* /dev/hdb is bigger than /dev/hda, so (backwards) rsync incrementals
are done to /backup/incremental, which uses the left over space on
/dev/hdb.
* Filesystems on /dev/hdb are backed up to tape using "dump". This is
cool, because the system can be up and running (and busy) while this
is happening, since the filesystems on /dev/hdb aren't active.
The 2nd script:
* Cleans up the incrementals, so there's always a certain percentage
of free space. When it needs space it removes one or more whole
days worth of incrementals.
Note that in this script there is *some* sanity checking to try and
ensure that sensible stuff is removed. However, you should always
*BEWARE* of scripts that run as root and automatically recursively
remove directories.
Just to be sure, I've changed the line that is supposed to remove
stuff so that it looks like:
echo "Not currently executing this command: rm -rf" "${this}"
so it now has to be modified so that the "rm -rf ..." is actually
activated... :-)
Remember, no warranty and all that...
peace & happiness,
martin
--------8<---------8<-------- CUT HERE --------8<---------8<--------
#!/bin/sh
# Uncomment these for debugging.
#set -x
DEBUG=echo
# Backup disk /dev/hda to /dev/hdb and then backup /dev/hdb to tape:
# 1. Using rsync, backup each mounted filesystem /X to
# /backup/current/X, which is mounted especially for the task. Old
# version of files are stored under
# /backup/incremental/DATETIME/X/...
# 2. Each mounted filesystem /X has its corresponding
# /backup/current/X backed up using dump. The /incremental
# filesystem is also dumped at the end.
# Note that the incrementals aren't currently cleaned up. Once we get
# an idea of the size of incrementals, we can started removing any
# directories older than (say) 6 months.
incr=/backup/incremental
yip="${incr}/`date +'%Y/%m/%d/%H%M%S'`"
tape=/dev/ntape
filesystems=`mount -t ext2 | awk '{print $3}'`
do_rsync () {
$DEBUG /usr/bin/rsync -axH --delete \
--backup --backup-dir="${yip}$1" \
"$1/" "/backup/current$1"
}
do_mt () {
${DEBUG} /bin/mt -f "${tape}" "$1"
}
do_label () {
tmp="/tmp/bkup$$"
date > "${tmp}"
n=1
for i in $* ; do
echo "$n $i" >> "${tmp}"
n=`expr $n + 1`
done
$DEBUG /bin/dd if="${tmp}" of="${tape}" bs=512 conv=sync
#$DEBUG /bin/tar cvf "${tape}" "${tmp}"
do_mt eof
/bin/rm -f "${tmp}"
}
do_dump () {
${DEBUG} /sbin/dump -0 -b 32 -a -f "${tape}" "$1"
}
######################################################################
echo "Check backup partitions starting: `date`."
fscks="${incr}"
disks=""
tapes=""
for i in ${filesystems} ; do
case "$i" in
/var/spool/squid)
:
;;
/)
fscks="${fscks} /backup/current"
disks="${disks} /"
tapes="${tapes} /backup/current"
;;
*)
fscks="${fscks} /backup/current${i}"
disks="${disks} ${i}"
tapes="${tapes} /backup/current${i}"
esac
done
for i in ${fscks} ; do
${DEBUG} /sbin/fsck -a $i
if [ $? -gt 1 ] ; then
echo "fsck of backup filesystem $i failed. Quitting!"
fi
done
echo "Check backup partitions finished: `date`."
######################################################################
echo "Disk backup starting: `date`."
${DEBUG} mount ${incr}
for i in ${disks} ; do
b="/backup/current${i}"
if ${DEBUG} mount "${b}" ; then
echo " ${i}"
do_rsync "${i}"
${DEBUG} umount "${b}"
else
echo "*** Failed to mount backup volume \"${b}\""
fi
done
${DEBUG} umount ${incr}
echo "Disk backup finished: `date`."
######################################################################
echo "Tape backup starting: `date`."
do_mt rewind
# Eek! Tape drive doesn't do hardware compression.
#do_mt datcompression
do_label ${tapes}
for b in ${tapes} ; do
if ${DEBUG} mount "${b}" ; then
echo " ${b}"
do_dump "${b}"
${DEBUG} umount "${b}"
else
echo "*** Failed to mount backup volume \"${b}\""
fi
done
do_mt offline
echo "Tape backup finished: `date`."
# Last line.
--------8<---------8<-------- CUT HERE --------8<---------8<--------
#!/bin/sh
# Cleanup the incremental backups on /backup/incremental to create
# free space.
base=/backup/incremental
threshold=90
mount $base
dirs=`echo /backup/incremental/*/*/*`
percent_used=`df /backup/incremental/ | tail -1 | sed -e 's/^.*[^0-9]\([0-9][0-9]*\).*$/\1/'`
while [ \( "${percent_used}" -gt ${threshold} \) -a -n "$dirs" ] ; do
this=`echo $dirs | sed -e 's/ .*$//'`
dirs=`echo $dirs | sed -e 's/^[^ ]* *//'`
case "$this" in
${base}/*/*/*)
echo "Removing old backup directory ${this}."
echo "Not currently executing this command: rm -rf "${this}"
;;
*)
echo "ERROR!!! Bad name for backup directory: ${this}."
dirs=""
esac
percent_used=`df /backup/incremental/ | tail -1 | sed -e 's/^.*[^0-9]\([0-9][0-9]*\).*$/\1/'`
done
if [ "${percent_used}" -gt ${threshold} ] ; then
echo "WARNING!!!"
echo "${base} has ${percent_used}% used, which is greater than limit of ${threshold}%"
echo "but there are no backups to remove!!!"
fi
umount $base
--------8<---------8<-------- CUT HERE --------8<---------8<--------
More information about the rsync
mailing list