Compression error? -- "Inflate (token) returned -5"

[Brian K. White]

I have sample data that exposes this repeatably:
http://lists.samba.org/archive/rsync/2008-October/021889.html

If you place old/file.dat on a remote box and then try to overwrite it 
with new/file.dat using rsync -az and any other options except --no-z or 
-W, it will fail almost immediately, every time.

If I retry the exact same command but simply remove -z, or add -W, it 
will work.

If I delete file.dat from the remote machine, sending new/file.dat will 
also work normally with or without -z

The error happens before 10 megs of data have been sent, and the files 
are each just over 1G in size, yet trying the same tests using only the 
beginning 100 or even 200 megs of the same files (extracted with dd) 
does not trigger the error, so I couldn't reduce the test data below 2G.
At least it's highly compressible data and the tar.bz2 is only 70M

Is this useful to anyone for finally having an example to debug and fix 
up rsync to detect and handle this situation instead of crashing?

I probably have currently at least about 4 more examples on various 
boxes so it happens more often than extremely rarely at least to me. One 
of those examples I already had to fix and I didn't grab copies of the 
files before fixing but they were even a little bigger at about 1.2G 
each. As I get around to fixing the others I'll try to see if I can find 
a smaller dataset that reproduces the compression/checksum bug.

I can give the sample data to pretty much anyone that wants it for a 
legitimate purpose like this, but it's a customer's data, not especially 
sensitive or useful, but I still can't publish it completely publicly.
I'll have to direct anyone who emails me back directly to a download 
url. This list is publicly archived so I don't feel good posting it here.

I've tarred up the two files and included a txt file with clear simple 
steps to exibit both the problem with -z and the fact that the same data 
without -z, or with -W, has no problem.

-- 
bkw