preserving Mac OS X metadata in rsync backups and restores

Moritz Heckscher moritz.heckscher at gmx.de
Sat Jan 19 03:39:59 GMT 2008


Am 2008-01-19 um 03:11 schrieb Matt McCutchen:
> On Fri, 2008-01-18 at 17:14 +0100, Moritz Heckscher wrote:
>> Another problem I'm thinking about is that rsnapshot should be run on
>> the server to "pull" the backups over the network. One cannot run it
>> on the clients and "push" the data to the server over the network --
>> which is what I'd prefer because I plan to not leave the server on
>> all day but rather have it woken up by the (laptop) clients when
>> needed who'll take care of the scheduling of the backups (using
>> anachron or launchd etc.) One could, however, run rsnapshot on the
>> clients to backup onto a locally attached storage device.
>
> I would like to dispel once and for all the myth that rsnapshot cannot
> be used with a push.  While it is /designed/ to pull backups, with  
> a few
> extra steps one can push a backup to an rsnapshot installation on
> another machine.  I outlined the setup here:
>
> http://lists.samba.org/archive/rsync/2007-December/019470.html

Good news indeed (for me at least)! Having read this FAQ on the  
rsnapshot page in an early stage, without having much background info  
about rsync and rsnapshot myself, I thought it was best to abandon  
the idea of pushed backups with rsnapshot altogether:

Q: Can I set the snapshot_root to a remote SSH path? I want to push  
my backups to a remote server, rather than pull them from a remote  
server.

A: Currently this is not possible. This would be a nice feature, but  
several questions remain unanswered:
* How can the integrity of the snapshot root be guaranteed if one or  
more remote servers have write access to it?
* When snapshots are rotated, which of the potentially several remote  
servers connecting is responsible for performing this task?
* One possibility is to have a "staging" area for files to be  
transferred to, then have rsnapshot sync from this staging area into  
the snapshot root. Can this be accomplished without taking up double  
the disk space?

Your suggestion does indeed take care of the "doubled disk space"  
issue nicely. It's only really possible with your very recent patch  
regarding the --no-tweak-hlinked option, so maybe now this FAQ entry  
could be update with the above link to your description! (By the way,  
would this always remain a patch or eventually go into the normal  
source?)

Regarding the other two points: If I have several clients, I guess I  
should have several independent deamons, each with "max connections"  
set to 1, and several independent "incoming" and "snapshots"  
directories, to avoid conflicts:

/backup/
     rsnapshot.laptop0.conf
     kick-rsnapshot-laptop0
     rsnapshot.laptop1.conf
     kick-rsnapshot-laptop1
     ./incoming/
         ./laptop0/
         ./laptop1/
     ./snapshots/
         ./laptop0/
         ./laptop1/

Could there be any problems if laptop0 and laptop1 are both pushing  
at the same time? Configurations are independent, and in that case  
there should simply be two instances of rsnapshot running, right?

> How are the clients going to wake the server?  Does the server wake up
> on any incoming network connection (I hope)?

Yes, that's the plan. I intent to have the clients send a wake-on-lan  
magic packet to wake the server, wait a little and then try to push  
the backup.

By they way, for the client to see if the backup operation succeeded,  
this could be easily tested by looking at the return code from the  
communication with the daemon, right?

>> 2) If I mount the server as a network drive on the clients using AFP,
>> SMB/CIFS, NFS, ..., and then backup to this 'locally attached' drive
>> with rsync (using rsnapshot), will I lose the metadata because of the
>> transfer via the SMB/... layer?
>
> I would strongly discourage you from pushing the backup via a network
> drive.  In addition to potential metadata mangling, you lose the  
> benefit
> of rsync's delta-transfer algorithm.

Yes, losing the metadata this way is likely to happen. I'll test it  
once my hardware is ready, just to see what would happen.

Just out of curiosity: Why would rsync not use the delta-transfer in  
that case? Is it because this would be the "local disk -> local disk"  
transfer case for which a fast connection is assumed, so no resources  
are wasted on trying to find differences?

> Matt

Well, thank you so very much. That post you've referred to was really  
one of the most helpful texts for me! Thank you also for the nice  
patch and all the hard work. I very much appreciate it!

-Moritz




More information about the rsync mailing list