rsync and hfs+ compressed files damaged

[Robert DuToit]

Hi Mike and All,

Pertaining to the corrupted files via rsync:

Build on OSX 10.6.4
rsync 3.0.6

patch -p1 <patches/fileflags.diff
patch -p1 <patches/crtimes.diff
patch -p1 <patches/rsync_3.0.6-hfs-compression_20091027.diff
./configure
make

(I have tried also using backup-dir-dels.diff and your extended_stats.diff with same results)


this is my own basic clone line
pathToPatchedRsync  -aHAXNx --fileflags --force-change --protect-decmpfs --stats -v --progress  /  /Volumes/Snowy  &> output file

if I just copy the /usr folder to some other directory it has the same effect.
pathToPatchedRsync  -aHAXNx --fileflags --force-change --protect-decmpfs --stats -v --progress  /usr  /Volumes/Snowy/testFolder  &> output file

I ran find_sl_damaged_files tool which checks the file size and the decmpfs and resource fork flags and it shows this:

/Volumes/Snowy/usr/bin/bzip2recover appears to be damaged
/Volumes/Snowy/usr/bin/file appears to be damaged
/Volumes/Snowy/usr/bin/chudRemoteCtrl appears to be damaged
/Volumes/Snowy/usr/bin/chumAddRights appears to be damaged
/Volumes/Snowy/usr/bin/ci appears to be damaged
/Volumes/Snowy/usr/bin/xed appears to be damaged
/Volumes/Snowy/usr/share/man/man1/ibtool.1 appears to be damaged
/Volumes/Snowy/usr/share/zoneinfo/America/Santa_Isabel appears to be damaged

I tested /Volumes/Snowy/usr/bin/file  and indeed it is broken on the that clone

It copies everything fine - seems a good clone but there are these several files. Someone has reported more corrupted files, or none, and say sometimes it changes from clone to clone. My experience produces the same corrupted files every time. I have done maybe 20 tests now. I'll ask others to check it out.

Now here's the strange part. If I copy just those problem files themselves, and not the entire /usr directory they aren't corrupted. Also if I copy a handful of exec files (including the problem ones) to another folder and copy those with rsync, they are fine too! As I said before I have read the Ars Technica article and others about hfs+ compression and seen reports of these same problems on a fresh install of 10.6

As a test I tried using CCC making a clone and then just the /usr folder and both times it comes up clean in the test - no corrupted files in there. I also tried ditto with the new −-hfsCompression option and that seems ok. So you must have done something right though I think we are using the same hfs+ patch that you posted. Or maybe you didn't use the patch but patched rsync directly….

Let me know if I can supply any more details on this.







On Jun 25, 2010, at 4:03 PM, Mike Bombich wrote:

> Can you share the steps required to reproduce this?  I've seen two reports of this but have not been able to reproduce it myself, and the users that reported it also could not reproduce it.
> 
> Mike
> 
> On Jun 25, 2010, at 5:16 AM, Robert DuToit wrote:
> 
>> Hi All,
>> 
>> I have been using rysnc 3.0.6 with Mike's rsync_3.0.6-hfs-compression_20091027.diff patch (as well as the standard osx patches) with good results. I discovered that clones done on 10.6 using this rsync cause some executable files ( the ones that are mostly compressed this way on osx ) to be damaged.
>> 
>> I did a test twice and indeed a handful of them are damaged. They all appear to be in /Volume/Snowy/usr/bin and/or /Volume/Snowy/usr/share.
>> 
>> I ran xattr on it and 
>> 
>> robert-dutoits-macbook-pro:~ astrid$ xattr -l /Volume/Snowy/usr/bin/file
>> com.apple.ResourceFork: 
>> com.apple.decmpfs: 
>> 
>> These attributes are supposed to be hidden when on 10.6. The files have 0 bytes to appear to be compressed but can not be un-compresed. I tried file on the clone and it doesn't work. Someone else reported du not working so there are some pretty basic files damaged. This appears to be the same issue that happens with some people when installing Snow Leopard fresh - some execs are damaged in this way. 
>> 
>> There are some threads about this, specifically bugs in the Snow Leopard causing this very thing to happen. 
>> 
>> There is a tool someone created for scanning for these damaged files at:
>> 
>> http://www.charlessoft.com/find_sl_damaged_files.zip
>> 
>> 
>> After running rsync 3.0.6 , with patches for hfs+, it reports 
>> 
>> /Volume/Snowy/usr/bin/chudRemoteCtrl appears to be damaged
>> /Volume/Snowy/usr/bin/chumAddRights appears to be damaged
>> /Volume/Snowy/usr/bin/ci appears to be damaged
>> /Volume/Snowy/usr/bin/bzip2recover appears to be damaged
>> /Volume/Snowy/usr/bin/file appears to be damaged
>> /Volume/Snowy/usr/bin/xed appears to be damaged
>> 
>> These same files are fine on the source volume.
>> 
>> Anyone know why this is happening? The hfs+ compressed files are otherwise handled perfectly using Mike's patch.
>> 
>> Thanks Rob
>> 
>> 
>> 
>> 
>> 
>> -- 
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>