rsync of STDIN to a file.
Matt McCutchen
matt at mattmccutchen.net
Thu Nov 19 08:38:57 MST 2009
On Thu, 2009-11-19 at 13:14 +0000, Mark Young wrote:
> I'm hoping rsync could be told to capture STDIN and treat it as a
> single file that it would then sync with the named file at the remote
> end. Perhaps something like this:
>
> tar czf - --files-from $FILE_LIST | rsync - $HOST:${BACKUP_FILE}
This could probably be implemented as a simple extension of the current
--copy-devices functionality. My only concern is that if the
delta-transfer algorithm makes a mistake, rsync will want to try the
transfer again, but stdin can only be consumed once. This could be
fixed by hacking in support for forking a subprocess to generate the
source content, or by using as the source a FUSE-based filesystem that
exposes the virtual tar file.
Another option would be to use rdiff(1). It can read from stdin, and I
don't think it attempts to retry if it makes a mistake.
To expand upon Ryan's remarks, even if you have an old compressed
tarball on the destination, the delta-transfer algorithm might not find
any matches since compression introduces non-local effects. To solve
that, the "gzip-rsyncable" patch (to gzip) would be needed. It was
maintained with rsync for a while but was apparently discontinued around
3.0.0pre6.
--
Matt
More information about the rsync
mailing list