bug? -z option and large compressed data

[Yasuoka Masahiko]

Hi,

I think I found a bug in usage of zlib.  rsync 2.5.6 with -z fails
like bellow.

  % cp install-disk2.iso /var/tmp/install-disk2.iso
  install-disk2.iso    100% |*****************************|   316 MB    00:56    
  % rsync -vIz install-disk2.iso 127.0.0.1:/var/tmp/install-disk2.iso
  install-disk2.iso
  deflate on token returned 0 (16384 bytes left)
  rsync error: error in rsync protocol data stream (code 12) at token.c(288)

"install-disk2.iso" is a CD-ROM image of "Turbo Linux Installer" and
this CD-ROM includes many gzip/bzip2 compressed datas(RPMs).

Let's looked at token.c.

    280      tx_strm.next_in = (Bytef *) map_ptr(buf, offset, toklen);
    281      tx_strm.avail_in = toklen;
    282      tx_strm.next_out = (Bytef *) obuf;
    283      tx_strm.avail_out = MAX_DATA_COUNT;
    284      r = deflate(&tx_strm, Z_INSERT_ONLY);
    285      if (r != Z_OK || tx_strm.avail_in != 0) {
    286              rprintf(FERROR, "deflate on token returned %d (%d bytes left)\n",
    287                      r, tx_strm.avail_in);
    288              exit_cleanup(RERR_STREAMIO);
    289      }

MAX_DATA_COUNT is 16383 and toklen may be 16384.  So, when deflate()'s
outputs data length is bigger than input, then above code will fail.
Normally output data length is smaller than input, but it is not true
to already compressed data.

--yasuoka
    Yasuoka Masahiko(yasuoka at yasuoka.net)

***
"gdb output"

(gdb) run --version
Starting program: /home/yasuoka/datacube/src/bin/rsync/rsync --version
rsync  version 2.5.6  protocol version 26
Copyright (C) 1996-2002 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, 
              IPv6, 64-bit system inums, 64-bit internal inums
	:
(gdb) b token.c:285
(gdb) condition 1 tx_strm.avail_in != 0 && tx_strm.avail_out == 0
(gdb) run -zI install-disk2.iso 127.0.0.1:/var/tmp/install-disk2.iso 
so 127.0.0.1:/var/tmp/install-disk2.iso

Breakpoint 1, send_deflated_token (f=7, token=2731, buf=0x8099150, 
    offset=44744704, nb=0, toklen=16384)
	:
(gdb) p tx_strm.avail_in
$1 = 16384
(gdb) p tx_strm.avail_out
$2 = 0
(gdb) p tx_strm.total_in 
$3 = 44744704
(gdb) p tx_strm.total_out
$4 = 44758356
(gdb) n
286                             rprintf(FERROR, "deflate on token returned %d (%d bytes left)\n",
(gdb) 
deflate on token returned 0 (16384 bytes left)
288                             exit_cleanup(RERR_STREAMIO);
(gdb) 
rsync error: error in rsync protocol data stream (code 12) at /home/yasuoka/datacube/src/bin/rsync/../../auxcmd/rsync/token.c(288)

Program exited with code 014.

***
patch for rsync 2.5.6

Index: token.c
===================================================================
RCS file: /vol/cvs/datacube/src/auxcmd/rsync/token.c,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 token.c
--- token.c	8 Apr 2002 08:35:30 -0000	1.1.1.1
+++ token.c	27 Jun 2003 11:24:25 -0000
@@ -125,7 +125,8 @@ static void simple_send_token(int f,int 
 #define TOKEN_REL	0x80	/* + 6-bit relative token number */
 #define TOKENRUN_REL	0xc0	/* ditto with 16-bit run count */
 
-#define MAX_DATA_COUNT	16383	/* fit 14 bit count into 2 bytes with flags */
+/* must be bigger than block_size + deflate()'s overhead */
+#define MAX_DATA_COUNT	CHUNK_SIZE
 
 /* For coding runs of tokens */
 static int last_token = -1;