[Bug 14081] New: --copy-command option for specifying custom file copying behaviour
samba-bugs at samba.org
samba-bugs at samba.org
Fri Aug 9 12:51:17 UTC 2019
https://bugzilla.samba.org/show_bug.cgi?id=14081
Bug ID: 14081
Summary: --copy-command option for specifying custom file
copying behaviour
Product: rsync
Version: 3.1.3
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: P5
Component: core
Assignee: wayne at opencoder.net
Reporter: samba at haravikk.com
QA Contact: rsync-qa at samba.org
This proposal is for the addition of an option enabling fully custom copying
behaviour in rsync, i.e- leveraging rsync primarily for detection of changes.
Custom copying behaviour is added using the --copy-command option, which takes
a variable number of arguments in a similar same style to the `find` command's
`-exec` option, terminated with a semi-colon.
For local copying, the special arguments `{src}` and `{dest}` can be used, and
will be substituted for absolute paths for the source file and its destination
respectively. For example:
rsync --copy-command cp {src} {dest} \; "$source" "$destination"
To perform a completely pointless copy using `cp` to absolutely no advantage.
When dealing with a remote source, the copy command will receive the file's
data as standard input. When dealing with a remote target, the copy command
should produce file data as standard output.
Custom copying behaviour can be useful in a number of situations where copying
tools have space-saving features, but do not have change detection or filtering
options as flexible as `rsync` does.
Note: the custom behaviour is *only* triggered when rsync determines that a
file's data should be copied, simple attribute, ownership and permission
changes would occur as normal, without invoking the custom command. Likewise,
directories etc. will be handled as normal. When a file is to be copied, rsync
will skip its normally attempt to find differences with the target, and instead
invoke the custom copy command, but will still synchronise any basic attributes
(owner, permissions etc.).
For a useful example, on macOS there are some particular use cases. One invokes
the `ditto` command to trigger compression of a file when transferred to an
APFS or HFS+ target, which could be taken advantage of like so:
rsync --copy-command ditto --hfsCompression {src} {dest} \; "$source"
"$destination"
Since this enables a form of transparent compression, rsync should still see
the copied file as identical to the original. Alternatively, the
--noHfsCompression option could be used instead to ensure that compressed files
are decompressed, even if the target would have been compatible.
Another useful macOS example is the use of cloning within an APFS volume, which
will produce an instantaneous copy requiring no additional space, and can be
used like so:
rsync --copy-command cp -c {src} {dest} \; "$source" "$destination"
Again, since the copy is a clone it should be recognised as identical to the
source by rsync on future passes.
In more complex cases, the copy command may produce a file that rsync cannot
compare normally, for example, if the file is passed through `gz` or `xz`, in
which case the use of checksums and size for comparison will need to be set
accordingly.
While it would be nice to see cloning and HFS compression features added
directly to rsync, this seemed like a much more flexible alternative, as it
also potentially enables the use of various compression and/or encryption tools
during transfers as well.
--
You are receiving this mail because:
You are the QA Contact for the bug.
More information about the rsync
mailing list