full upload happening even though only a timestamp has changed
jw schultz
jw at pegasys.ws
Tue Apr 22 11:04:05 EST 2003
On Tue, Apr 22, 2003 at 12:36:02AM +0000, Tom Goulet wrote:
> > If the mtime indicates the file has changed it will do an
> > update. That doesn't mean that any actual file data will be
> > transfered.
>
> That's what I thought was supposed to happen, but that's not what's
> happening. My testing transfer has finished now and I'll show the
> transcript here:
> tomg at nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg
> building file list ...
> 46349 files to consider
> wrote 1358812 bytes read 20 bytes 44551.87 bytes/sec
> total size is 5240249974 speedup is 3856.44
> tomg at nova:~/mylibrary$ touch /extra/library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso
> tomg at nova:~/mylibrary$ rsync -rtlPvessh /extra/library teep:/extra/tomg
> building file list ...
> 46349 files to consider
> library/software/Knopper,_Klaus/KNOPPIX_V3.2-2003-04-18-EN.iso
> 729716736 100% 1.02MB/s 0:11:19
> wrote 731164664 bytes read 36 bytes 1005037.39 bytes/sec
> total size is 5240249974 speedup is 7.17
> tomg at nova:~/mylibrary$
>
> As you can see, the second use of Rsync wrote 729805852 more bytes than
> the first.
The first time it didn't update the iso file.
> > If all the blocksums match it will effectively do a local copy to the
> > temp file and then rename back. If you want it to do a whole-file
> > checksum as part of decideing whether to update you will need to use
> > the -c option which overrides mtime.
>
> I don't understand much of that. Do I have to use the -c command line
> option whenever I do not want Rsync to rely solely on the modification
> time? I'm pretty confused. I'm certain I could use Rsync without that
> option to make sure the remote file was right without actually uploading
> file data, it's not like I wouldn't notice two days of upload time.
Normally rsync uses file size and mtime to decide whether to
skip a file. Generating checksums is expensive so that is
not done unless updating the file. If you use -c it will
generate a file checksum on each end to decide whether to
skip instead of comparing mtime.
There is a known problem with files of this size where block
checksums fail on the first pass. When that happens there is
a message that would look something like "redoing
filename(1524)" in stdout. It is possible you are
encountering this problem and -P is hiding it. Try
redirecting stdout, don't use -P and perhaps add an extra
-v.
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync
mailing list