Unchanged file recopied + meaning of speedup (was: Is this correct?)

John Van Essen vanes002 at umn.edu
Tue Dec 30 12:31:34 EST 2003


(I changed the Subject to something less generic...)

On Mon, 29 Dec 2003, kevin_david_carpenter <unknown at crengland.com> wrote:
>
> I am trying to copy an 8 gig file. After first copying it over I try
> and rsync it again. There are no changes so why is it copying the
> file over again? Here is the output of what I am trying to do. And
> what does speedup mean?

See the --modify-window option.  It may apply in your case.

The speedup value is the ratio of total size to sum of bytes read and
written.  It represents the savings in I/O from rsync's intelligent
synchronization where it (by default) transfers a file only if it does
not exist on the receiver, or does exist but has a different size or
timestamp (and in those cases it can compare block checksums to avoid
copying unchanged portions of a file).

The speedup value is roughly the multiplier to apply to the number of
actual bytes tranferred to get the number of bytes that a really dumb
"full file, every file" synchronization would have transferred.

On an initial rsync, the speedup will be near 1.00 (no speedup).
Subsequent rsyncs will have speedups that depend on how much data had
to be copied.  Note that in your second transfer, where it did have to
resend the file, it was able to match all 8 GB of data and transferred
only a small fraction of that in matching checksum values.  Hence a
speedup of 1600+.
-- 
        John Van Essen  Univ of MN Alumnus  <vanes002 at umn.edu>



> [root at VMWARE Storage]# rsync --verbose --recursive -t --stats --
> progress ntapps.dsk vmware2::home/
> [root at VMWARE Storage]# rsync --verbose --recursive -t --stats --
> progress ntapps.
> building file list ...
> 1 file to consider
> ntapps.dsk
>   8388608512 100%    2.11MB/s    1:03:11
> rsync[8049] (sender) heap statistics:
>   arena:         147992   (bytes from sbrk)
>   ordblks:            3   (chunks not in use)
>   smblks:             0
>   hblks:              0   (chunks from mmap)
>   hblkhd:             0   (bytes from mmap)
>   usmblks:            0
>   fsmblks:            0
>   uordblks:       42648   (bytes used)
>   fordblks:      105344   (bytes free)
>   keepcost:       11168   (bytes in releasable chunk)
> 
> Number of files: 1
> Number of files transferred: 1
> Total file size: 8388608512 bytes
> Total transferred file size: 8388608512 bytes
> Literal data: 8388608512 bytes
> Matched data: 0 bytes
> File list size: 37
> Total bytes written: 8389632637
> Total bytes read: 56
> 
> wrote 8389632637 bytes  read 56 bytes  2212164.19 bytes/sec
> total size is 8388608512  speedup is 1.00
> 
> [root at VMWARE Storage]# rsync --verbose --recursive -t --stats --
> progress ntapps.dsk vmware2::home/
> building file list ...
> 1 file to consider
> ntapps.dsk
>   8388608512 100%    2.35MB/s    0:56:40
> rsync[8820] (sender) heap statistics:
>   arena:         147992   (bytes from sbrk)
>   ordblks:            5   (chunks not in use)
>   smblks:             0
>   hblks:              1   (chunks from mmap)
>   hblkhd:        266240   (bytes from mmap)
>   usmblks:            0
>   fsmblks:            0
>   uordblks:       59056   (bytes used)
>   fordblks:       88936   (bytes free)
>   keepcost:       11168   (bytes in releasable chunk)
> 
> Number of files: 1
> Number of files transferred: 1
> Total file size: 8388608512 bytes
> Total transferred file size: 8388608512 bytes
> Literal data: 0 bytes
> Matched data: 8388608512 bytes
> File list size: 37
> Total bytes written: 2048125
> Total bytes read: 3072062
> 
> wrote 2048125 bytes  read 3072062 bytes  1307.67 bytes/sec
> total size is 8388608512  speedup is 1638.34




More information about the rsync mailing list