DO NOT REPLY [Bug 4162] New: Wanted: a mechanism to prevent rsync
network compression of compressed files
samba-bugs at samba.org
samba-bugs at samba.org
Wed Oct 11 05:57:03 GMT 2006
Summary: Wanted: a mechanism to prevent rsync network compression
of compressed files
OS/Version: Mac OS X
AssignedTo: wayned at samba.org
ReportedBy: rsync at name99.org
QAContact: rsync-qa at samba.org
rsync, of course, offers the -z flag to allow for the transfer of compressed
file over the network during a backup.
My experience using this flag has been that (on a 1GHz PPC laptop connected
- without using the -z flag the average data rate over the connection is about
2.5MB/s which is about where you'd expect 802.11g to max out, all things
- with using the -z flag the average data rate over the connection is about
1MB/s, and the CPU is maxed out.
Now, if what was being transferred was a stream of text (compression ratio of
say 4 or so), this would still be a win. But on a modern personal system, the
bulk of the material transferred (by bytes, not by file number) is going to be
photos, audio files and video files, ie already compressed stuff, so the mean
compression rate over the entire stream of data is going to be just a bit over
1, and using -z is a loss.
The obvious issue, then, is how can we get the goodness of -z for text files,
while avoiding the cost of the CPU to compress files that aren't going to
Two obvious strategies spring to mind:
* We could track the progress of the compression and bail out if it is less
than some lower limit, maybe 1.2 or so. Maybe run compression till about 8KiB
into the file, see how things are going and coontinue compressed or switch to
* We could simply allow for a user-supplied list of files, (presumably the same
syntax as backup_excludes) that we would not bother to try to compress. This
may not be as robust a strategy as the first scheme, but it is much easier to
program, and should be good enough for most purposes. I'd recommend in addition
that rsync ship with a starter template for this file that includes all the
usual suspects from *.gzip through *.mp3, *.mov, *ogg etc etc.
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
More information about the rsync