preserving Mac OS X metadata in rsync backups and restores

David Miller millerdc at fusion.gat.com
Fri Jan 18 19:56:12 GMT 2008


On Jan 18, 2008, at 8:14 AM, Moritz Heckscher wrote:

> Hello all,
>
> I'm new to the list, but have done quite a bit of researching before  
> regarding the support of Mac OS X specific features (resource forks,  
> extended attributes, ACLs, file creation & modification date).
>
> By reading the archives, I get the impression that the current  
> version of rsync 3.0.0pre8 is quite far in this respect. At least it  
> sounds so, and I thank the developers very much for this! I like  
> your approach much more than the (very buggy) one originally pursued  
> by Apple (store metadata in separate ._ file).
>

Be careful and test, test, test. I tried using pre8 to sync two local  
Xserve RAID's(about 2TB of data) and I'm seeing these errors.

rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken  
pipe (32)
[receiver] internal abbrev error!
rsync error: error in rsync protocol data stream (code 12) at  
xattrs.c(565) [receiver=3.0.0pre8]
rsync: connection unexpectedly closed (175959 bytes received so far)  
[sender]
rsync error: error in rsync protocol data stream (code 12) at  
io.c(600) [sender=3.0.0pre8]

I have another Xserve RAID(about 1.3TB) and I don't get those errors  
when syncing with pre8. I'm trying to pin down what files/folders are  
causing the problem now.

> I plan to do the following:
>
> * Run a Linux server (Ubuntu, I guess, on a ext3 partition) with two  
> separate internal ATA hard disks formatted in XFS and configured in  
> software RAID to store the actual backup data. (As I understand, I  
> should use XFS rather than ext3 because XFS supports extended  
> attributes large enough to hold also larger converted Mac resource  
> forks.)
>
> * Back up from different Mac OS X clients (cuurently all on 10.4,  
> but I might upgrade them to 10.5 later) to the server using rsync  
> over ssh. This should hopefully preserve (most of) the Mac-specifif  
> metadata on the server. (Actually I plan to use rsnapshot, but I  
> believe if I have rsync installed in the newest version and possibly  
> tell rsnaphot to use the appropriate rsync options, things will be  
> the same.)
>
> Now my question is the following:
>
> 1) What would I have to do to ensure the metadata is also restored  
> correctly? I assume I will have to use rsync for restoring also, and  
> if I just copy over data (using, e.g, scp or over an AFP or CIFS or  
> NFS network mount), I will lose this metadata. Is this correct?
>

Why not use rsync3 for both backup and restore. Either use ssh (rsync - 
azXA --delete /path/to/source server:/path/to/target) or setup an  
rsync daemon server. This way you let rsync handle the metadata.



> Another problem I'm thinking about is that rsnapshot should be run  
> on the server to "pull" the backups over the network. One cannot run  
> it on the clients and "push" the data to the server over the network  
> -- which is what I'd prefer because I plan to not leave the server  
> on all day but rather have it woken up by the (laptop) clients when  
> needed who'll take care of the scheduling of the backups (using  
> anachron or launchd etc.) One could, however, run rsnapshot on the  
> clients to backup onto a locally attached storage device.
>

You don't need rsnapshot. Use the --link-dist option to create  
incremental backups.

> This leads me to the second question:
>
> 2) If I mount the server as a network drive on the clients using  
> AFP, SMB/CIFS, NFS, ..., and then backup to this 'locally attached'  
> drive with rsync (using rsnapshot), will I lose the metadata because  
> of the transfer via the SMB/... layer?
>
> Thanks a lot for a great program!
> -Moritz
>
> -- 
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


David Miller.


More information about the rsync mailing list