[Bug 10552] Sender checksum calculation significantly slower with compression enabled

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=10552

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED
           Severity|normal                      |enhancement

--- Comment #3 from Wayne Davison <wayned at samba.org> 2014-04-19 16:57:45 UTC ---
Yes, this is part of the way that rsync trades CPU (and disk I/O) to reduce
transfer I/O.  When compressing, both sides of the connection "prime the pump"
for a compressed file's transfer by including matching data in the compression
stream.  This ensures that by the time a difference is found that the data will
compress more optimally.  It can't know in advance that there will be no
differences in the whole file, since by the time it finds that out the transfer
is done.  If this is causing you problems, you might try --checksum, but that
can be slower too if there are a lot of unchanged files in the transfer that
match in size & mtime, though you can improve the speed of --checksum though
one of the rsync patches that caches checksum data.

There has been some thought to disable the shared-data part of the compression,
since it complicates the compression-lib usage and (as you saw) is sometimes
wasteful.  I'm marking this as an enhancement request to add such a compression
mode.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.