clarifiaction about rync zlib.

Matt McCutchen hashproduct+rsync at gmail.com
Thu Oct 12 21:53:10 GMT 2006


On 10/12/06, thomas david  clarke <cam30724 at port.ac.uk> wrote:
> i would like a more detailed answer to a question.
> {Z_INSERT_ONLY is used to update  history}
>
> How is this feature used in rsync?

My understanding is as follows; Wayne, please correct me as necessary.
 Zlib compresses essentially by noticing regions of input that
duplicate earlier regions and replacing them with back references.
When rsync uses the incremental transfer algorithm, the sender sends
the receiver a combination of literal data and information about
matched blocks.  It would be great if Zlib could back-reference
matched data when compressing literal data, even though the matched
data is not actually sent over the connection.  To this end, the
sender passes the entire file to Zlib, but it sets Z_INSERT_ONLY when
passing the matched data so Zlib knows to omit that data from the
compressed output.

The description of --compress in the man page alludes to this: "Note
that this option typically achieves better compression ratios than can
be achieved by using a compressing remote shell or a compressing
transport because it takes advantage of the implicit information in
the matching data blocks that are not explicitly sent over the
connection."

> Other than transfering less data, what other aplications might this feature have?

Transferring less data is the only advantage, but Z_INSERT_ONLY could
provide this advantage to programs other than rsync.  It would be
useful to any program that wants to take advantage of data that the
other side already has when compressing new data.

Matt


More information about the rsync mailing list