Rsync compression problem - sometimes ineffective?

[Matt McCutchen]

On Thu, 2008-06-12 at 13:35 -0400, Bodle, Donald E wrote:
> Using "bytes sent"/"literal data" from statistics as a rough estimation
> (I know there is overhead in the bytes sent) of the effectiveness of
> compression, most days I see reasonable compression

> My initial thought was that days of no apparent compression were when
> the majority of the changed files were small files (like when gzipping a
> small ASCII file doubles it size) or already compressed files.  But so
> far  I haven't been able to confirm this.  I'm not sure this logic
> applies since rsync compresses data blocks (at least as I understand
> it), and those blocks would be fairly consistent in size (I think).  Is
> this general understanding of rsync's compression correct?

My guess is that the files are already compressed.

To see the actual size (compressed if applicable) of the delta rsync is
sending for each file, use the %b log option, e.g.,
--out-format='%b %i %n%L' .  You can compare those numbers with and
without compression to see which deltas aren't compressing as well as
you expect.  Unfortunately, %b only seems to work on a run that really
updates a destination, so you'll have to use a throwaway destination
(perhaps with --compare-dest to the real one) for the tests; %b ought to
work in --only-write-batch mode.  To investigate why a particular delta
isn't compressing, you could use rdiff to write the delta to a file and
then look at the data inside.

Matt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/rsync/attachments/20080612/fc8be79d/attachment.bin