rsync script for snapshot backups
linux at thehobsons.co.uk
Sun Jun 19 17:27:04 UTC 2016
Dennis Steinkamp <dennis at lightandshadow.tv> wrote:
> i tried to create a simple rsync script that should create daily backups from a ZFS storage and put them into a timestamp folder.
> After creating the initial full backup, the following backups should only contain "new data" and the rest will be referenced via hardlinks (-link-dest)
> Well, it works but there is a huge flaw with his approach and i am not able to solve it on my own unfortunately.
> As long as the backups are finishing properly, everything is fine but as soon as one backup job couldn`t be finished for some reason, (like it will be aborted accidently or a power cut occurs)
> the whole backup chain is messed up and usually the script creates a new full backup which fills up my backup storage.
Yes indeed, this is a typical flaw with many systems - you often need to throw away the partial backup.
One option that comes to mind is this :
Create the new backup in a directory called (for example) "new" or "in-progress". If, and only if, the backup completes, then rename this to a timestamp. If when you start a new backup, if the in-progress folder exists, then use that and it'll be freshened to the current source state.
Also, have you looked at StoreBackup ? http://storebackup.org
I does most of this automagically, keeps a definable history (eg one/day for 14 days, one/week for x weeks, one/30d for y years), plus it keeps file hashes so can detect bit-rot in your backups.
More information about the rsync