Permissions causing full backups?

[Matt McCutchen]

Link,

I discovered a way that you can have your Windows machine push to an
rsync daemon and still have rsnapshot manage the backups so you don't
have to mess around with backup numbers and link-dest yourself.  In
short, give the daemon a "post-xfer exec" that invokes rsnapshot.  (CC
to the rsync list because others might wish to use this setup.)

Setting this up requires some fiddling and testing on the Linux server.
First, add a post-xfer exec line to call a script ./kick-rsnapshot, like
this:

[bkups]
        path = bkups
        read only = false
        auth users = <all the users>
        secrets file = bkups.secrets
        post-xfer exec = ./kick-rsnapshot

Then, create the following kick-rsnapshot script and make it executable:

#!/bin/bash
if [ "$RSYNC_EXIT_STATUS" == "0" ]; then
        rsnapshot -c rsnapshot.conf occasional
fi

Note that rsnapshot is only called if the push is successful.  If the
push gets interrupted, you can just keep trying, and rsnapshot will
store the backup when you eventually succeed.  If "file is vanished" is
a problem, you can change the if:

if [ "$RSYNC_EXIT_STATUS" == "0" ] || [ "$RSYNC_EXIT_STATUS" == "24" ]; then

Now, write the configuration file rsnapshot.conf.  If your system has
a /etc/rsnapshot.conf.default, you may wish to copy it to the daemon's
directory and edit from there.

Choose a snapshot root inside the daemon directory, _different_ from the
module directory.  The default log and lock files are in system-wide
areas; either change them to be inside the daemon directory or comment
them out.  Set up a single backup interval; I used "occasional".
Finally, create a single backup entry as follows:

backup  /path/to/daemon/bkups/   ./      rsync_long_args=--link-dest=../../bkups/

This entry copies the module contents into the snapshot.  Since the
module is also specified as a link-dest directory, all files except
directories get hard-linked.

Now test the setup by pushing to the rsync daemon.  Since the backup
management is done on the Linux server, the command on Cygwin is a
one-liner:

rsync <options> /cygdrive/c/Documents and Settings/$USERNAME/My
Documents/My Music/ $user@$bkup_ip::bkups/

(Note the double colon, w

Rsync on cygwin connects directly to the rsync daemon and uploads the
files to the module.  Then the daemon tells rsnapshot on the Linux
server to convert the module to a snapshot.  Check to see whether an
occasional.0 with the user's files has appeared on the Linux server.

As long as a file remains unchanged, its appearances in the module and
in all the snapshots will be hard-linked together.  When the source file
changes, the rsync daemon writes a new destination file and moves it
over the old hard-linked one, so old backups are not corrupted.

So that's how you would accommodate a single user, but you have multiple
users.  Give each user his/her own module.  You can put all the
usernames and passwords in the same secrets file and mention that file
once at the top; you can also move the "read only" and "post-xfer exec"
lines to the top.  Then, give each module a separate "auth users" line
naming just the user who will be pushing to the module.

I'm guessing only one user pushes at a time, and each push creates a
snapshot.  Do you want each snapshot to contain (a) the data for all
users or (b) just the user who pushed?  For (a), simply add a separate
"backup" entry for each module to rsnapshot.conf.

I think (b) makes more sense, but it takes more work.  You need a
separate rsnapshot installation for each user with its own configuration
file and snapshot root, and then you should make kick-rsnapshot examine
$RSYNC_MODULE_NAME and/or $RSYNC_USER_NAME to determine which rsnapshot
installation to kick.

If you want to change an rsnapshot setting, you shouldn't have to modify
each configuration file separately.  You might want to write a template
configuration file with placeholders like USER.  Then you can do
something like this in kick-rsnapshot:

sed -e "s/USER/$RSYNC_USER_NAME/g" rsnapshot.conf.template | rsnapshot -c /dev/stdin occasional

Devious, isn't it?

By the way, rsnapshot doesn't seem to like my version of cp.  rsnapshot
calls "cp -al occasional.0/ occasional.1/", but cp refuses to create a
destination directory named with a trailing slash.  If you have this
problem, write a script "mycp" that removes the slash (below) and
specify it as cmd_cp in rsnapshot.conf.

#!/bin/bash
/bin/cp -al $2 ${3%/}

> Do I run this on the samba server at startup so that it's running all
of the time?  Or, am I supposed to start it each time I want to make a
backup?  It seems to need to be attended (supply password).

Have the daemon start (call ./start) when the system starts and stop
(call ./stop) when the system shuts down.  The Windows user pushing her
files does need to enter her rsync password, or she can store it in a
file and give rsync --password-file=<file>.

I realize this setup procedure is complicated, but I think the resulting
system will work well.  If you need help carrying it out, perhaps we
could go on AIM (mattmccutchen) or I could SSH onto your machine and
guide you in the ytalk shell.  Enjoy!

-- Matt McCutchen