Protocol stream error copying extended attribute, silent failure to copy all data

[Matt Rosenberg]

I'm using rsync 3.0.7 on Mac OS X 10.6, compiled with HFS+ enhancements
according to Mike Bombich's instructions at
http://www.bombich.com/rsync.html. Rsync repeatedly exits with a protocol
data stream error on some com.apple.FinderInfo extended attributes. While
testing this issue, I found that rsync is not copying all extended
attributes even when there is no error message. I'm using a folder of fonts
as an example, but I have experienced the protocol error when copying other
data. This seems like a huge bug, and in my experience those often turn out
to be operator error. Apologies if I waste anyone's time.

In this example, I'm using rsync to make a copy from scratch of the source
data: rsync307 -aX --delete Licensed\ Fonts /Volumes/Storage/

I repeatedly get this error:

[sender] internal abbrev error on Licensed Fonts/Postscript/bradley
(com.apple.FinderInfo, len=32)!
rsync error: error in rsync protocol data stream (code 12) at xattrs.c(636)
[sender=3.0.7]

If you repeat the command a few times, sometimes the error is:

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken
pipe (32)
rsync: connection unexpectedly closed (16827 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(601)
[sender=3.0.7]

In all my testing, I have only seen the error occur with a
com.apple.FinderInfo attribute, and only for a directory.

Running rsync with sudo gives the same error.

The system has no problem reading the extended attribute from the source
folder, or using the xattr command to write it to the target folder:

matt$ xattr -l Licensed\ Fonts/Postscript/bradley/
com.apple.FinderInfo:
00000000  03 99 01 5D 04 7C 02 2F 07 A0 5A 50 00 01 02 07
|...].|./..ZP....|
00000010  FF F8 FF F0 C3 40 00 00 00 00 00 00 00 01 6C 28
|..... at ........l(|
00000020
matt$ xattr -l /Volumes/Storage/Licensed\ Fonts/Postscript/bradley/
matt$ xattr -wx com.apple.FinderInfo "`xattr -px com.apple.FinderInfo
Licensed\ Fonts/Postscript/bradley/`" /Volumes/Storage/Licensed\
Fonts/Postscript/bradley/
matt$ xattr -l /Volumes/Storage/Licensed\ Fonts/Postscript/bradley/
com.apple.FinderInfo:
00000000  03 99 01 5D 04 7C 02 2F 07 A0 5A 50 00 01 02 07
|...].|./..ZP....|
00000010  FF F8 FF F0 C3 40 00 00 00 00 00 00 00 01 6C 28
|..... at ........l(|
00000020

After manually setting the extended attribute, the rsync command completes
without error messages. The process is repeatable, always choking on the
bradley folder.

After rsync completed without error messages, I compared the sizes of source
and copy:

matt$ du -k -d1 /Installers/Licensed\ Fonts/
245196    /Installers/Licensed Fonts//Adobe Font Folio - OpenType Edition
51800    /Installers/Licensed Fonts//OpenType
256756    /Installers/Licensed Fonts//Postscript
43308    /Installers/Licensed Fonts//TrueType
598692    /Installers/Licensed Fonts/
matt$ du -k -d1 /Volumes/Storage/Licensed\ Fonts/
245196    /Volumes/Storage/Licensed Fonts//Adobe Font Folio - OpenType
Edition
51800    /Volumes/Storage/Licensed Fonts//OpenType
246540    /Volumes/Storage/Licensed Fonts//Postscript
38528    /Volumes/Storage/Licensed Fonts//TrueType
583696    /Volumes/Storage/Licensed Fonts/

My understanding is that the sizes should match. The number of items in each
is the same:

matt$ du -a /Installers/Licensed\ Fonts/ | wc -l
   11916
matt$ du -a /Volumes/Storage/Licensed\ Fonts/ | wc -l
   11916

I saved the output of 'du -a' for source and copy, then ran a diff. There
are hundreds of differences, and it appears that many extended attributes
were not copied. Here's a typical example:

matt$ ls -lh Licensed\ Fonts/TrueType/CCZoinks/CCZoinks.t1
-rwx------@ 1 matt  matt     0B Jul 16  1999 Licensed
Fonts/TrueType/CCZoinks/CCZoinks.t1
matt$ ls -lh Licensed\ Fonts/TrueType/CCZoinks/CCZoinks.t1/..namedfork/rsrc
-rwx------  1 matt  matt   9.5K Jul 16  1999 Licensed
Fonts/TrueType/CCZoinks/CCZoinks.t1/..namedfork/rsrc

matt$ ls -lh /Volumes/Storage/Licensed\ Fonts/TrueType/CCZoinks/CCZoinks.t1
-rwx------@ 1 matt  matt     0B Jul 16  1999 /Volumes/Storage/Licensed
Fonts/TrueType/CCZoinks/CCZoinks.t1
matt$ ls -lh /Volumes/Storage/Licensed\
Fonts/TrueType/CCZoinks/CCZoinks.t1/..namedfork/rsrc
-rwx------  1 matt  matt     1B Jul 16  1999 /Volumes/Storage/Licensed
Fonts/TrueType/CCZoinks/CCZoinks.t1/..namedfork/rsrc

The file is actually a PostScript Type 1 font, where all the content is in
the resource fork. Rsync created a com.apple.ResourceFork attribute on the
new file, but the attribute is empty.

After this discovery, I ran my original rsync command again, then checked
sizes:

matt$ du -kd1 /Volumes/Storage/Licensed\ Fonts/
245196    /Volumes/Storage/Licensed Fonts//Adobe Font Folio - OpenType
Edition
51800    /Volumes/Storage/Licensed Fonts//OpenType
254488    /Volumes/Storage/Licensed Fonts//Postscript
42216    /Volumes/Storage/Licensed Fonts//TrueType
595332    /Volumes/Storage/Licensed Fonts/

More data was copied, but it's still missing some. Overall, it took 6 runs
of the same rsync command to produce a copy that's the same size as the
original.

Is this a bug? Should I be using different flags in my rsync command?

Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20110131/f0be45ea/attachment.html>