plain source -> encrypted destination: rsync + gpg (--dest-filter patch)

jw schultz jw at
Thu Jun 26 23:42:22 EST 2003

On Thu, Jun 26, 2003 at 04:31:29PM +0200, addady at wrote:
> > If i recall correctly (i've not gone back to review that
> > patch) that patch required whole-file transfers two avoid
s/two/to/  #ick
> > doing the operation twice per file.  If the filter is
> > deterministic and particularly if changes have minimal
> > cascade effect you may be able to still take advantage of
> > the rsync algorithm.
> You recall correctly.
> It is not clear to me how using "whole-file" and still take advantage of
> the rsync algorithm.

You meant "how to use"?  The --whole-file option disables
the rsync algorithm.

> The idea of this patch seem to me very useful.
> I'm plan to use it with the "rsyncable" gzip version
> (
> The "whole-file" is not necessary is this case.  I think that the patch need
> to be modifiy so that
> whole-file will be a option.

If the receiver does the filtering whole-file is necessary.
To show why i will describe what it takes if the receiver
does the filtering without whole-file assuming gzip is the

	gunzip to temp file to generate block checksums.*

	delete temp file.

	---- interval when other files may be processed ----

	gunzip to temp file* again and merge file with blocks
	from sender.

	delete temp file.

	gzip resultant file.

Notice that this also requires a reverse filter.

If the filtering is done on the sender.

	gzip file to temp.*

	match blocks or send whole gziped file to receiver.

	delete temp file.

Observe the sender-side only requires a forward filter.

*The temp file is needed because match and block
checksumming use memory mappings of the file so i don't
think it is capable of reading from a pipe.  Merging even
does seeks because matched blocks may be out of order.

Whether the rsync algorithm is useful depends on the
determinism and change-cascade effects of the filter.
A ignore-size option would have to be created to deal with
filters that alter the file length.  The --checksum option
would be best not used as it would require every file be
filtered during flist generation and then changed files
filtered again for sending.

	J.W. Schultz            Pegasystems Technologies
	email address:		jw at

		Remember Cernan and Schmitt

More information about the rsync mailing list