preserving Mac OS X metadata in rsync backups and restores

[Victor Shoup]

The backup-bouncer tool looks very helpful.
I tried testing my own backup solution (see http://shoup.net/xattr )  
with backup-bouncer.
It did fine -- it preserves everything on cross-platform backups,  
except for those things
it was not designed to: ACLs (I don't know much about these) and BSD  
flags other than uchg, uappnd
(I modified the backup-bouncer test suite to restrict to just these  
BSD flags, to verify).

   -- Victor Shoup


On Jan 18, 2008, at 11:12 PM, Moritz Heckscher wrote:

> Am 2008-01-19 um 01:37 schrieb Moritz Heckscher:
>> 3) To help with the checking/comparing, someone else built a  
>> little tool. It not only creates a collection of test files but  
>> can also compare the original and transferred versions afterwards:
>>
>> <http://www.n8gray.org/blog/2007/04/27/introducing-backup-bouncer/>
>
> I have downloaded the tool and compiled it on my Mac with 10.4 to  
> do some testing. In this case, that's transfers using rsync  
> 3.0.0pre8 from one HFS+ disk image to another HFS+ image (on a  
> computer with an HFS+ disk as well). Later I'd be using cross- 
> platform cross-filesystem transfers, so the results below should  
> present something like the "upper limit" of what is currently  
> possible.
>
> Using the following options for rsync:
>
> --archive --hard-links --acls --xattrs --executability --numeric-ids
>
> I get the following output from backup-bouncer:
>
>
> ------------------ rsync3.0.0pre8 ------------------
> This copier exited with error code 23
> This copier produced log output in:
>    /Volumes/Dst/21-rsync3.0.0pre8/log
> Verifying:    basic-permissions ... ok
> Verifying:           timestamps ...
>    Sub-test:    modification time ... ok
> ok
> Verifying:             symlinks ... ok
> Verifying:    symlink-ownership ... ok
> Verifying:            hardlinks ... ok
> Verifying:       resource-forks ... ok
> Verifying:         finder-flags ... ok
> Verifying:         finder-locks ... FAIL
> Verifying:        creation-date ... FAIL
> Verifying:            bsd-flags ... FAIL
> Verifying:       extended-attrs ...
>    Sub-test:             on files ... ok
>    Sub-test:       on directories ... ok
>    Sub-test:          on symlinks ... ok
> ok
> Verifying: access-control-lists ...
>    Sub-test:             on files ... ok
>    Sub-test:              on dirs ... ok
> ok
> Verifying:                 fifo ... FAIL
> Verifying:              devices ... FAIL
> Verifying:          combo-tests ...
>    Sub-test:  xattrs + rsrc forks ... ok
>    Sub-test:     lots of metadata ... ok
> ok
>
>
> So that looks not so bad actually. The main problem for me are the  
> mangled creation dates. However, not even Apple's own cp with  
> options -pR (-p to "preserve in the copy as many of the  
> modification time, access time, file flags, file mode, user ID, and  
> group ID as allowed by permissions") passes this test:
>
>
> ------------------ cp-apple ------------------
> Verifying:    basic-permissions ... ok
> Verifying:           timestamps ...
>    Sub-test:    modification time ... ok
> ok
> Verifying:             symlinks ... ok
> Verifying:    symlink-ownership ... FAIL
> Verifying:            hardlinks ... FAIL
> Verifying:       resource-forks ... ok
> Verifying:         finder-flags ... ok
> Verifying:         finder-locks ... ok
> Verifying:        creation-date ... FAIL
> Verifying:            bsd-flags ... ok
> Verifying:       extended-attrs ...
>    Sub-test:             on files ... ok
>    Sub-test:       on directories ... ok
>    Sub-test:          on symlinks ... FAIL
> FAIL
> Verifying: access-control-lists ...
>    Sub-test:             on files ... ok
>    Sub-test:              on dirs ... ok
> ok
> Verifying:                 fifo ... ok
> Verifying:              devices ... ok
> Verifying:          combo-tests ...
>    Sub-test:  xattrs + rsrc forks ... ok
>    Sub-test:     lots of metadata ... ok
> ok
>
>
> For comparison, this is Apple's 'fixed' rsync  version 2.6.3   
> protocol version 28:
>
>
> ------------------ rsync-apple ------------------
> This copier exited with error code 24
> This copier produced log output in:
>    /Volumes/Dst/10-rsync-apple/log
> Verifying:    basic-permissions ... ok
> Verifying:           timestamps ...
>    Sub-test:    modification time ... ok
> ok
> Verifying:             symlinks ... ok
> Verifying:    symlink-ownership ... ok
> Verifying:            hardlinks ... ok
> Verifying:       resource-forks ... FAIL
> Verifying:         finder-flags ... FAIL
> Verifying:         finder-locks ... FAIL
> Verifying:        creation-date ... FAIL
> Verifying:            bsd-flags ... FAIL
> Verifying:       extended-attrs ...
>    Sub-test:             on files ... FAIL
>    Sub-test:       on directories ... FAIL
>    Sub-test:          on symlinks ... FAIL
> FAIL
> Verifying: access-control-lists ...
>    Sub-test:             on files ... FAIL
>    Sub-test:              on dirs ... FAIL
> FAIL
> Verifying:                 fifo ... ok
> Verifying:              devices ... ok
> Verifying:          combo-tests ...
>    Sub-test:  xattrs + rsrc forks ... FAIL
>    Sub-test:     lots of metadata ... FAIL
> FAIL
>
>
> Adding --fake-super to the options of rsync doesn't help:
>
>
> ------------------ rsync3.0.0pre8 ------------------
> This copier exited with error code 23
> This copier produced log output in:
>    /Volumes/Dst/21-rsync3.0.0pre8/log
> Verifying:    basic-permissions ... FAIL
> Verifying:           timestamps ...
>    Sub-test:    modification time ... ok
> ok
> Verifying:             symlinks ... ok
> Verifying:    symlink-ownership ... FAIL
> Verifying:            hardlinks ... ok
> Verifying:       resource-forks ... ok
> Verifying:         finder-flags ... ok
> Verifying:         finder-locks ... FAIL
> Verifying:        creation-date ... FAIL
> Verifying:            bsd-flags ... FAIL
> Verifying:       extended-attrs ...
>    Sub-test:             on files ... FAIL
>    Sub-test:       on directories ... FAIL
>    Sub-test:          on symlinks ... FAIL
> FAIL
> Verifying: access-control-lists ...
>    Sub-test:             on files ... FAIL
>    Sub-test:              on dirs ... FAIL
> FAIL
> Verifying:                 fifo ... FAIL
> Verifying:              devices ... FAIL
> Verifying:          combo-tests ...
>    Sub-test:  xattrs + rsrc forks ... FAIL
> cmp: ./many-metadata: Permission denied
>    Sub-test:     lots of metadata ... FAIL
> FAIL
>
>
> -Moritz
>
> -- 
> To unsubscribe or change options: https://lists.samba.org/mailman/ 
> listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart- 
> questions.html