rsync 1tb+ each day

Steve Bonds knnf6cy7w001 at
Wed Feb 5 09:56:01 EST 2003

On Tue, 4 Feb 2003, Kenny Gorman |Rsync List| wrote:

> My question is this:
> Is it possible to tell rsync to update the blocks of the target file
> 'in-place' without creating the temp file (the 'dot file')?

It does not look like this is possible.  In receiver.c around line 452 you
can see:

/* recv file data */
recv_ok = receive_data(f_in,buf,fd2,fname,file->length);

fd2 looks to always be a file descriptor generated via do_mkstemp.

> I can guarantee that no other operations are being performed on the
> file at the same time.  The docs don't seem to indicate such an
> option.

Rsync works best over low bandwidth links for which the disk I/O is
minimal.  You might try one of these ideas for your high-bandwidth
  + hack receiver.c so that receive_data uses fd1 (the original file)
    - also comment out finish_transfer, which does the rename and 
      sets the permissions.  If perms are important, then set them 
    - check the dest file closely to make sure it's not mangled
    - This is completely untested.  If you're not comfortable with 
      hacking and the usual followup debugging, then skip it.
  + compile and run rsync with profiling libraries to make sure that 
    it's slow where you think it's slow.  I've been surprised 
    before.  ;-)
  + use a "dumber" file transfer method (FTP, netcat) that will be 
    faster on beefier hardware.  Netcat is especially fast if you have
    a private network where you don't have to worry about pesky things
    like authentication.

For Oracle datafiles I've had excellent luck with a homebrew file transfer
that "compresses" blocks of zeros by sending a message that means "Hey!  
I just read XXX blocks of nothing but zeroes."  The receiver then creates
a sparse file on the destination for this area of zeroes.  It's very handy
for making copies of the database for read-only purposes and it saves lots
of disk space.  It doesn't help at all vs. something like netcat if your
datafiles are mostly full, tho.

  -- Steve

PS:  You can get info on netcat from:

More information about the rsync mailing list