preserving Mac OS X metadata in rsync backups and restores

[Robert DuToit]

Hi All,
  I ran the bouncer tests too and came up with same results except the  
"fifo" and "devices" failed. I ran it from command line as well as  
from "do shell script" command with same results. I used the same  
patches. Nevertheless, I am thrilled with the results here for OSX and  
can't thank you all enough for putting rsync 3 together. I did some  
incremental speed tests with my 16GB Home folder from my old powerbook  
G4:

from do shell script; destinations all to external firewire drive on  
OS 10.5.1

with patches:
initial backup 54 minutes
no-change-incremental backups  7  - 14 minutes

without patches:
initial backup    52 minutes
no-change-incremental backups 5 - 7 minutes

the patches don't make a huge difference for the initial backups but  
no-change-incrementals do seem to go on twice as long. Not sure why  
really- The output showed nothing unusual.  Things would go faster  
from the command line since I am running rsync 3 through my wrapper  
application with the "do shell script" command. . I changed one letter  
on some deeply nested files and they all showed up fine in the  
incremental snapshots.

By the way, after the initial 16+ GB copy, the incrementals took up an  
incredible .05 +- Gb of space. The apple supplieed rsync on Tiger and  
Leo both recopy a lot of data each time so the incrementals never  
worked well.

Thanks Again, Rob D




On Jan 20, 2008, at 4:17 PM, Mike Bombich wrote:

> With rsync 3 pre8 and Mac OS 10.5.1:
>
> bash-3.2# patch -p1 <patches/osx-create-time.diff
> bash-3.2# patch -p1 <patches/flags.diff
> bash-3.2# ./prepare-source
> bash-3.2# patch -p1 <patches/backup-dir-dels.diff
> bash-3.2# ./configure
> bash-3.2# make
>
> bash-3.2# ~/rsync-3.0.0pre8/rsync -aHAX --fileflags /Volumes/ 
> Source/ /Volumes/Target/
>
> bash-3.2# bbouncer verify -d /Volumes/Source /Volumes/Target
> Verifying:    basic-permissions ... ok
> Verifying:           timestamps ...
>   Sub-test:    modification time ... ok
> ok
> Verifying:             symlinks ... ok
> Verifying:    symlink-ownership ... ok
> Verifying:  symlink-permissions ... ok
> Verifying:            hardlinks ... ok
> Verifying:       resource-forks ... ok
> Verifying:         finder-flags ... ok
> Verifying:         finder-locks ... ok
> Verifying:        creation-date ... ok
> Verifying:            bsd-flags ... ok
> Verifying:       extended-attrs ...
>   Sub-test:             on files ... ok
>   Sub-test:       on directories ... ok
>   Sub-test:          on symlinks ... ok
> ok
> Verifying: access-control-lists ...
>   Sub-test:             on files ... ok
>   Sub-test:              on dirs ... ok
> ok
> Verifying:                 fifo ... ok
> Verifying:              devices ... ok
> Verifying:          combo-tests ...
>   Sub-test:  xattrs + rsrc forks ... ok
>   Sub-test:     lots of metadata ... ok
> ok
>
>
>
> As far as performance is concerned, I'm surprised how fast rsync  
> 3pre8 is for no-change-incrementals.  Incremental recursion alone  
> doesn't really explain it, I'm still trying to see if something is  
> fishy or if it really can be this good.
>
> Mike
>
> On Jan 18, 2008, at 10:38 PM, Moritz Heckscher wrote:
>
>>
>> Am 2008-01-19 um 05:31 schrieb Matt McCutchen:
>>
>>> On Sat, 2008-01-19 at 05:12 +0100, Moritz Heckscher wrote:
>>>> Using the following options for rsync:
>>>>
>>>> --archive --hard-links --acls --xattrs --executability --numeric- 
>>>> ids
>>>>
>>>> I get the following output from backup-bouncer: [...]
>>>> Verifying:            bsd-flags ... FAIL
>>>
>>> BSD flags are handled by flags.diff in rsync- 
>>> patches-3.0.0pre8.tar.gz .
>>>
>>>> Verifying:         finder-locks ... FAIL
>>>
>>> A request for enhancement was recently entered for finder locks:
>>>
>>> https://bugzilla.samba.org/show_bug.cgi?id=5188
>>>
>>> But Wayne says he thinks they are handled by flags.diff too.
>>>
>>>> Verifying:        creation-date ... FAIL
>>>
>>> Creation dates are handled by osx-create-time.diff .
>>
>> Thanks for mentioning those diffs. I'll try tomorrow and report here.
>>
>>>> Verifying:                 fifo ... FAIL
>>>> Verifying:              devices ... FAIL
>>>
>>> These are supposed to work with --archive, at least if rsync runs as
>>> root.  In any event, you probably don't care about these if you're  
>>> just
>>> backing up user data.
>>
>> Yes, I was expecting this to work. I did run as root (using sudo)  
>> and also tried --device and --specials, but these are already  
>> included in --archive. Hm, I'll run it again tomorrow, maybe I've  
>> messed up somehow (it's 05:30 in the morning...).
>>
>>>> Adding --fake-super to the options of rsync doesn't help:
>>>
>>> Naturally, because that option makes rsync wrap the source  
>>> metadata in
>>> an rsync-specific extended attribute that it sets on the destination
>>> file, whereas the checker tool is expecting the real destination
>>> metadata to be set.
>>
>> Yes, that absolutely makes sense. I shouldn't have included the  
>> output here. Obviously one would need a second "recover" step using  
>> rsync before checking with the tool.
>>
>>> Matt
>>
>> Thanks for your fast help, I'm learning a lot!
>> -Moritz
>> -- 
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>
>
> -- 
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html