compression of source and target files

[Shachar Shemesh]

Kenneth Simpson wrote:
> Chuck Wolber wrote:
>   
>> On Fri, 21 Sep 2007, Kenneth Simpson wrote:
>>
>>   
>>     
>>> Hi - there's a flag for rsync to compress the files in transit - is it 
>>> possible to compress one side (target) with gzip and have rsync still 
>>> work correctly?
>>>     
>>>       
>> It'll still work correctly, but compressing a compressed file can actually 
>> make it slightly bigger and wastes CPU cycles in the process.
>>
>> ..Chuck..
>>
>>   
>>     
>  Sorry, I neglected to mention the source is uncompressed but
> we need to compress the target file because we're running out
> of disk space and the files are highly compressible.
>
>
> We can't compress the source since the files are large and
> compressing the source would create other problems.
>  
> The original thought was to use a file system with compression (I
> think Linux has such a beast) but this would at least require a
> kernel rebuild which we won't be able to do for awhile.
>
> The second thought was that we might be able to gzip on the fly
> and have rsync work correctly (since it's compressing them in
> transit.)
>   
gzip, as is, will destroy rsync's ability to sync partial file changes.
Gzip does have, however, a patch that adds a "rsyncable" option to the
command line, that makes the compressed output rsync ready. The only
problem I see with your suggestion is that, as far as I know, rsync
cannot sync a stream of data to a file. Do have a look at librsync,
however, which reportedly can do that.

Shachar
>
>
>