Take2: rsync over ssh with --link-dest

[Matt McCutchen]

On Tue, 2009-11-10 at 22:47 -0500, Alex wrote:
> > If the daemon solution is looking fundamentally more complicated than
> > the set of forced server commands you previously had, you're probably
> > going about it the wrong way and I could help if you post the
> > rsyncd.conf file.
> 
> Okay, great, it's included below.

> use chroot = no
> syslog facility = local5
> 
> [cdrive]
>    list = true
>    path = /backup/servers/xxx.xxx.228/cdrive
>    comment = 228-cdrive
>    read only = true
>    gid = root
>    uid = root
[more modules like this...]

> The client side command looks something like this:
> 
> rsync -avz --delete --numeric-ids --exclude-from=/backup/exclude-file.txt \
>      --link-dest=/backup/seg/serverA/xxx.xxx.228/cdrive/20091109 \
>      rsync://serverA/cdrive/20091110/ \
>      /backup/seg/serverA/xxx.xxx.228/cdrive/20091110

Your daemon setup looks exactly right.  I don't understand why you
consider it more complicated than the previous setup.  Previously you
had one SSH authorized key for each backup point with the appropriate
path forced; now the daemon configuration contains the same information.
The daemon configuration looks more verbose, but that could be helped by
moving the parameters that are the same for all modules ("list", ...) to
the global section at the top, or you could even generate the
configuration file with M4.  The client command is the same except for
the specification of a module name instead of an SSH key.  Am I missing
something?

> > Realize that the ability to set a module's "path" in the rsyncd.conf 
> > file and have the client refer to the module by a well-known name gives
> > you the same functionality, and actually more, because the client can
> > specify a subdirectory path within the module.
> 
> I'm not sure I understand fully, but this may be what I'm already
> doing. An example would be helpful.

You are indeed doing this.  Your source is:

rsync://serverA/cdrive/20091110/

The rsync://serverA/cdrive/ is mapped to
/backup/servers/xxx.xxx.228/cdrive according to the configuration file,
resulting in:

/backup/servers/xxx.xxx.228/cdrive/20091110/

In contrast, the forced-server-command approach would clobber the
20091110/ part, preventing you from setting up the --link-dest properly.
Is it clear now?

> To make it even more complicated, on the receiving (client) side, I
> have to pass the backup script I use through sed to rewrite the date
> which gets appended to the source path, to update every day so I can
> use the --link-dest rsync argument to create a backup of only the
> changes from day-to-day, and hard link the rest from the reference
> location, which is usually the previous day.

It seems to me that you would have had to do the same thing under the
forced-server-command setup.  No?

In any case, sed is overkill to vary the date.  You can just use a shell
variable:

DATE=$(date +%Y%m%d)
rsync ... /backup/seg/serverA/xxx.xxx.228/cdrive/$DATE

-- 
Matt