rsync script for snapshot backups

[Simon Hobson]

Dennis Steinkamp <dennis at lightandshadow.tv> wrote:

> i tried to create a simple rsync script that should create daily backups from a ZFS storage and put them into a timestamp folder. 
> After creating the initial full backup, the following backups should only contain "new data" and the rest will be referenced via hardlinks (-link-dest) 
> ...
> Well, it works but there is a huge flaw with his approach and i am not able to solve it on my own unfortunately. 
> As long as the backups are finishing properly, everything is fine but as soon as one backup job couldn`t be finished for some reason, (like it will be aborted accidently or a power cut occurs) 
> the whole backup chain is messed up and usually the script creates a new full backup which fills up my backup storage. 

Yes indeed, this is a typical flaw with many systems - you often need to throw away the partial backup.
One option that comes to mind is this :
Create the new backup in a directory called (for example) "new" or "in-progress". If, and only if, the backup completes, then rename this to a timestamp. If when you start a new backup, if the in-progress folder exists, then use that and it'll be freshened to the current source state.

Also, have you looked at StoreBackup ? http://storebackup.org
I does most of this automagically, keeps a definable history (eg one/day for 14 days, one/week for x weeks, one/30d for y years), plus it keeps file hashes so can detect bit-rot in your backups.