rsync of STDIN to a file.

Matt McCutchen matt at mattmccutchen.net
Thu Nov 19 15:02:33 MST 2009


On Thu, 2009-11-19 at 17:29 +0000, Mark Young wrote:
> So if I understand you correctly you are suggesting that I modify my
> backup line to be something like this:
> 
> tar czf - --files-from $FILE_LIST | gzip --rsyncable > ${BACKUP_FILE}
> or:
> GZIP="--rsyncable" tar cvf - --files-from $FILE_LIST > ${BACKUP_FILE}
> 
> then
> rsync ${BACKUP_FILE} $HOST:${BACKUP_FILE}

Correct.

> I just tried some tests but didn't get the results I expected. I seem
> to be getting quite different files:
[...]
> $ tar cf m9.tar ./newdir
> $ tar cf m10.tar ./newdir
> $ sum -r m*tar
> 54744  3340 m10.tar
> 54744  3340 m9.tar
> $ gzip --rsyncable m9.tar
> $ gzip --rsyncable m10.tar
> $ sum -r m*tar
> 21783  2929 m9.tar.gz
> 30175  2929 m10.tar.gz
> 
> It seems that gzip produces different output even given the same
> input. Not very rsync friendly.

As Eliot said, this is gzip's metadata-saving behavior, which I suggest
you turn off with "-n".  Even so, it is not as harmful to delta-transfer
performance as you make it sound: it will only result in sending one
extra block over the network.

> I'm not sure I can see what I'd use the rdiff command for since rsync
> will do the delta-transfer cleverness for me.

Rdiff can read from stdin more easily than rsync can, as per your
original question.

-- 
Matt



More information about the rsync mailing list