compression and encryption

[Vaclav Dvorak]

Hello folks!

I've been thinking about adding the capability to store files compressed 
and/or encrypted on either side of the rsync transfer. Let's say there's 
a storage space provider. I want to store my files on that server 
compressed (so I don't use more paid space than I need), and encrypted 
(so they can't read my data).

I don't know much about encryption, but I suppose that there are some 
ciphers that are reasonably strong and don't have the same problem like 
gzip (that a single changed byte in the middle of the file affects the 
contents of the rest of the file)? Even if not, the same thing like with 
gzip (--rsyncable) could probably be done. The goal is to do the 
encryption on one side so that the storage provider doesn't ever see the 
unencrypted content or the key.

As for compression, I was thinking of making rsync temporarily 
uncompress the file, then update it, and then recompress it. This way, 
the gzip rsyncable patch wouldn't be needed, and one could actually use 
any other compressor, like bzip2.

One very general way to achieve things like this would be to add the 
ability to filter every file through a given filter before and after the 
transfer, like this:

rsync --filter-source="encrypt --key=xyzzy" --filter-dest-before=bunzip2 
--filter-dest-after=bzip2 /source/directory host:

It would be extremely slow, of course, but it may be worth it in some 
situations. (The --filter-source wouldn't actually need to be so slow if 
the filter was guaranteed to produce exactly as many bytes as it consumes.)

Of course, there's at least one major problem with this: generally, the 
server can't let the client run any command as the filter. But the 
server side rsync could be modified to have a config file option with a 
regexp for allowed commands, or something like that.

Obviously, rsync could also be amended with built-in encryption. Less 
general solution, but worth considering, too.

Any thoughts or opinions? Any existing work that I should be aware of? 
Would a patch in some of the directions outlined above be accepted?

Thanks,

Vaclav Dvorak