rsync to servers highly sensitive to IO load

[Jeff Woods]

Perfect!  It sounds like "--append" it the magic bullet in this case...

Thanks,
Jeff

Wayne Davison wrote:
> On Fri, Jul 11, 2008 at 10:10:16AM -0400, Jeff Woods wrote:
>   
>>     --bwlimit=17 --partial --append
>>     
>
> You only want to use --append if you can guarantee that the files will
> not have any changes in the existing data on the receiving side.  A
> modern rsync does not compute a full-file checksum for --append unless
> you use a second --append, since that slows down the appending.
>
> If you're using -c (--checksum), you probably don't want to do that,
> since that's super slow.  Just use -t (--times) and let the normal
> size+mtime check look over things for you.  If you find that you really
> need checksumming, you might want to look at the db.diff in the patches
> dir that lets you cache checksums in a DB (i.e. SQLite or MySQL) and
> associate them with unchanged files (since it matches a file's size,
> mtime, ctime, and inode, it is safe).
>
> If the source of the I/O is rsync's scanning of the directories (not the
> checksumming of the files), you may want to look into the slow-down.diff
> file in the patches dir, as that provides a way to get rsync to do its
> directory scanning more slowly.
>
>   
>> negating the need to re-read the entire file for a post-transfer
>> checksum
>>     
>
> There is no such thing in rsync, since it computes the checksum as the
> file is written.
>
> ..wayne..
>   



-----------------------------------------
The information contained in this e-mail message is intended only
for the personal and confidential use of the recipient(s) named
above. This message may be an attorney-client communication and/or
work product and as such is privileged and confidential. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and
that any review, dissemination, distribution, or copying of this
message is strictly prohibited. If you have received this
communication in error, please notify us immediately by e-mail, and
delete the original message.
-------------- next part --------------
HTML attachment scrubbed and removed