rsync and hfs+ compressed files damaged
Robert DuToit
rdutoit at comcast.net
Thu Jul 1 05:16:18 MDT 2010
Hi Mike and All,
I have been doing some research on the damaged HFS compressed files and found some interesting clues. Looking at mike's patch so far I can't find any reason for the damage (I have to learn more C language ) but what still seems odd is that the damage only happens when copying the whole /usr directory or /usr/bin /usr/share. Individually copying one of the original files does not create the damaged file.
Pertaining to the use of Mike's patch on OSX 10.6.4 copying /usr/bin and resulting damaged files
In /usr/bin/ I found that the originals of the damaged files are different from the other "normal" exec files. In this case I am looking at the "find" exec.
If I copy "find" from /usr/bin/ to the desktop via finder copy, it is changed to resemble the "normal" execs. And indeed if I replace the original with this copy in /usr/bin and copy /usr/bin with patched rsync, the copied "find" is no longer damaged!
So it appears that when the patched rsync encounters one of these "odd" files ( and only when copying the whole /usr/bin/ directory containing them ) it causes the problem.
Here are results using hfsdebug (sorry about the amount of reading!)
####################
original "file" exec - works ok but different from other execs in /usr/bin. ( It shows the com.apple.decmpfs flag and data is stored in Resource Fork)
flags = 0000000000000110
. File has extended attributes.
# Data Fork
logicalSize = 0 bytes
# Resource Fork
logicalSize = 101755 bytes
totalBlocks = 25
# Attributes
<Attributes B-Tree node = 6754 (sector 0x294b0)>
# Attribute Key
keyLength = 46
pad = 0
fileID = 3904426
startBlock = 0
attrNameLen = 17
attrName = com.apple.decmpfs
# Inline Data
recordType = 0x10
reserved[0] = 0
reserved[1] = 0
attrSize = 16 bytes
attrData = 66 70 6d 63 04 00 00 00 b0 a0 03 00 00 00 00 00
f p m c
compression magic = cmpf
compression type = 4 (resource fork has compressed data)
uncompressed size = 237744 bytes
####################
A normal exec "find" in /usr/bin/ ( data stored in # Data Fork so it is showing as uncompressed in finder)
flags = 0000000000000010
# BSD Info
ownerID = 0 (root)
groupID = 0 (wheel)
= 0
# Data Fork
logicalSize = 147680 bytes
totalBlocks = 37
clumpSize = 2445
# Resource Fork
logicalSize = 0 bytes
####################
the original "file" exec now copied via patched rsync within the whole /usr/bin directory (Note: no data in Data Fork or Resource Fork)
First note that the xattrs show up ( not supposed to when seen from 10.6)
robert-dutoits-macbook-pro:~ astrid$ xattr -l /Users/astrid/Desktop/copied_bin_test/bin/file
com.apple.ResourceFork:
com.apple.decmpfs:
and the hfsdebug report:
flags = 0000000000000110
. File has extended attributes.
# Data Fork
logicalSize = 0 bytes
# Resource Fork
logicalSize = 1 bytes
totalBlocks = 1
# Attributes
<Attributes B-Tree node = 5258 (sector 0x23730)>
# Attribute Key
keyLength = 46
startBlock = 0
attrNameLen = 17
attrName = com.apple.decmpfs
# Inline Data
recordType = 0x10
reserved[0] = 0
reserved[1] = 0
attrSize = 1 bytes
attrData = 04
compression magic = .
compression type = 44000000 (?)
uncompressed size = 1224979098644812920 bytes
####################
I can't put this all together yet but if anyone can see what is going on I would much appreciate it. What throws me though it that the damage only happens when copying the whole /usr/bin/ directory…
Again - this is rsync 3.0.6 patched with fileflags, crtimes and Mike's hfs patch.
Thanks, Rob
On Jun 25, 2010, at 4:03 PM, Mike Bombich wrote:
> Can you share the steps required to reproduce this? I've seen two reports of this but have not been able to reproduce it myself, and the users that reported it also could not reproduce it.
>
> Mike
>
> On Jun 25, 2010, at 5:16 AM, Robert DuToit wrote:
>
>> Hi All,
>>
>> I have been using rysnc 3.0.6 with Mike's rsync_3.0.6-hfs-compression_20091027.diff patch (as well as the standard osx patches) with good results. I discovered that clones done on 10.6 using this rsync cause some executable files ( the ones that are mostly compressed this way on osx ) to be damaged.
>>
>> I did a test twice and indeed a handful of them are damaged. They all appear to be in /Volume/Snowy/usr/bin and/or /Volume/Snowy/usr/share.
>>
>> I ran xattr on it and
>>
>> robert-dutoits-macbook-pro:~ astrid$ xattr -l /Volume/Snowy/usr/bin/file
>> com.apple.ResourceFork:
>> com.apple.decmpfs:
>>
>> These attributes are supposed to be hidden when on 10.6. The files have 0 bytes to appear to be compressed but can not be un-compresed. I tried file on the clone and it doesn't work. Someone else reported du not working so there are some pretty basic files damaged. This appears to be the same issue that happens with some people when installing Snow Leopard fresh - some execs are damaged in this way.
>>
>> There are some threads about this, specifically bugs in the Snow Leopard causing this very thing to happen.
>>
>> There is a tool someone created for scanning for these damaged files at:
>>
>> http://www.charlessoft.com/find_sl_damaged_files.zip
>>
>>
>> After running rsync 3.0.6 , with patches for hfs+, it reports
>>
>> /Volume/Snowy/usr/bin/chudRemoteCtrl appears to be damaged
>> /Volume/Snowy/usr/bin/chumAddRights appears to be damaged
>> /Volume/Snowy/usr/bin/ci appears to be damaged
>> /Volume/Snowy/usr/bin/bzip2recover appears to be damaged
>> /Volume/Snowy/usr/bin/file appears to be damaged
>> /Volume/Snowy/usr/bin/xed appears to be damaged
>>
>> These same files are fine on the source volume.
>>
>> Anyone know why this is happening? The hfs+ compressed files are otherwise handled perfectly using Mike's patch.
>>
>> Thanks Rob
>>
>>
>>
>>
>>
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
More information about the rsync
mailing list