[Bug 9864] New: Allow permanent compression of destination files

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=9864

           Summary: Allow permanent compression of destination files
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: me at haravikk.com
         QAContact: rsync-qa at samba.org


Rsync already supports compression for improving transfer speed, which is
great, but it would also be nice to see the option for permanent compression to
reduce space on the target file-system.

Here is an example of what options might look like:

--dest-compress - enables compression of files on the destination.
--dest-compress-level - sets the compression level for files on the
destination. Any value from 1-9 implies --dest-compress, while 0 disables it.
If this parameter is not supplied then --compress-level will be used if set,
otherwise default compression is used.
--dest-compress-ext - an optional extension to append to compressed files, e.g
- 'gz'. This extension is checked when checking for files that already exist on
the destination, so that compressed files are not deleted.
--source-compress-ext - specifies an extension to be checked for on the source,
targeting files that need to be decompressed before sending. This is useful in
cases where a destination structure created with compression needs to be synced
to another target in a decompressed form. Where possible decompression should
be skipped if transfer compression or destination compression are also enabled
(i.e - a delta update is not being performed).

In keeping with ticket #8915:
--dest-compress-minsize - the minimum size of uncompressed file that
compression will be attempted upon, e.g - if a file is under 4mb then
compression may be skipped.
--dest-compress-maxsize - the maximum size of uncompressed file that
compression will be attempted upon, e.g - skip compression on files larger than
1gb to save processing.


The implication of this feature of course is that a sync/backup destination can
be a lot smaller than the source destination, allowing you to get more out of
the space; especially handy on common file-systems such as ext4 that do not
natively support compression.

However, this saving is at the cost of processing, as a file on the destination
may need to be decompressed on-the-fly in order to compare checksums, and
changes cannot be applied via --inplace or --append. If a file is new to the
destination however then if --compress-level and --dest-compress-level are the
same then it should be possible to simply write the file straight to disk in
the compressed form; however if any of the compression details conflict then a
gzip -d optionally followed by a gzip may be required to remove and
(optionally) re-apply compression.

Of course the savings won't be as good as a single compressed archive would be,
but it retains the freedom of a synced structure but with some of the space
requirement reduced.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.