Data loss (corruption with rsync)

[Matt McCutchen]

On Sat, 2009-12-05 at 00:19 +0200, kordex - wrote:
> Discussion in enlightened irc channel (OFTC/#ck) gave possible clue
> for solving this, although it's just a quess:
> 
> 0006 @      con || KordeX >4GB file?
> 0007 +   KordeX || not all of those are
> 0007 +   KordeX || i mean most of them are <100MB
> 0007 @      con  | yes but that's the reason it failed
> 0008 @      con  | rsync uses the rzip algorithm which was 32bit limited as I
>           discovered whenI was updating lrzip
> 0012 +   KordeX || con > it uses rzip even when not using any --compress?
> 0012 +   KordeX || or does it use it when using append-verify or fuzzy options?
> 0012 @      con  | I'm not sure but it uses the rzip for the diff function I
>           think
> 0013 +   KordeX || oh that's the reason i was affected
> 0013 +   KordeX || i think i should re: re: my email with that update
> 0013 @      con  | actually I'm guessing
> 0014 +   KordeX || well i add this anyways, give one clue to them
> 0014 @      con  | dont quote me on that, but I had all sorts of troubles
>           dealing with > 2^32 sized files with the rzip code
> 0014 @      con  | and I had to rewrite most of the rzip algorithm to be 64bit
>           smart
> 0014 @      con  | which unfortunately also slowed it down quite a lot
> 0015 @      con  | unless you were working on >4GB files that is, where it sped
>           it up

Rsync and rzip do not share code, though the algorithms may be similar.
I don't believe a 2^32-byte limit exists in the current version of
rsync.

-- 
Matt