rsync to tape

[Martin Pool]

On  7 Aug 2001, Sudarshan Ramaswamy <sudar at synopsys.COM> wrote:
> Hi
> 
> I could do that anyways.  I  am interested in this option  for the following
> 
> 1) When I run rsync in cron. It copies the data to a directory. If it can
> copy the difference of files regularly to a tape
> I could use the disk space on my machine for something else.

rsync can't do that, but librsync is meant to be able to.  

I haven't tried this yet, but here's a *sketch* of how it should
work.

 1. Read the section about rsync-based tape backup in tridge's thesis,
    linked from the rsync site

 2. Download and build the rdiff tool from librsync
    <http://rproxy.samba.org/>

 3. To make a full backup:

    mknod /tmp/hash.pipe p
    rdiff signature /tmp/hash.pipe /var/state/full.rsig &
    tar c |
	 tee /tmp/hash.pipe | 
	 dd of=/dev/tape0

    We need the fifo so that the tar file can be split and used both
    for generating a signature and also written to tape.  The
    signature is relatively small and will be used to generate delta
    tapes later.

 4. To make a delta backup:

    tar c |
	rdiff delta /var/state/full.rsig | 
	dd of=/dev/tape0

 5. To restore from the delta backup, you need to have both the delta
    and full backups accessible at the same time, either by copying
    the delta to disk, or by having two tape drives, or something
    else.  Then, with the full backup in tape0

    rdiff patch /dev/tape0 /dev/tape1 |
	tar x

    or

    (insert delta)
    dd if=/dev/tape0 of=/tmp/delta.rpatch
    (insert full)
    rdiff patch /dev/tape0 /tmp/delta.patch |
	tar x

    It would be nice if you could first restore the full backup, and
    then apply the delta on top, using the filesystem basically as
    temporary storage for the full version.  Something like this:

    gunzip </dev/tape0 | tar x
    ; swap tapes to the delta
    tar c | rdiff patch - /dev/tape0 | tar x

    The problem with this is that untarring and retarring may not
    generate a byte-for-byte identical file, which is required to make
    the delta restore correctly.  inode numbers will be different, and
    the atimes may be different.  So if you have a format which obeys
    this property you may be OK.

    It's not necessarily worth worrying about, since assuming
  
      full backup >>> delta backup >>> system ram

    this approach will require sweeping over the whole filesystem
    twice, which is probably much slower than finding a place to
    temporarily hold the delta.

    You can compress the deltas and full backups by inserting
    appropriate pipes to gzip.  You need for the tarballs not to be
    compressed when they pass through rdiff or the deltas will not be
    very efficient.  Eventually rdiff will do compression itself, but
    that's not implemented yet.

I hope you found this useful or at least amusing.  I'd be interested
to hear whether it works :-)

-- 
Martin