full upload happening even though only a timestamp has changed
jw at pegasys.ws
Tue Apr 22 11:43:22 EST 2003
On Tue, Apr 22, 2003 at 01:32:54AM +0000, Tom Goulet wrote:
> > > As you can see, the second use of Rsync wrote 729805852 more bytes than
> > > the first.
> > The first time it didn't update the iso file.
> I realise that... My problem is that it shouldn't have updated the file
> the second time, since the remote file was identical to the local file.
> > Normally rsync uses file size and mtime to decide whether to
> > skip a file. Generating checksums is expensive so that is
> > not done unless updating the file. If you use -c it will
> > generate a file checksum on each end to decide whether to
> > skip instead of comparing mtime.
> Okay, I got all that. So if the modification time is different, Rsync
> should get a checksum, right?
Wrong. If the mtime is different rsync updates the file.
> And if the checksum is the same, Rsync
> should not upload, right? But in this case, Rsync is uploading even
> though the checksum should be is the same.
You are confusing upload with update.
> > There is a known problem with files of this size where block
> > checksums fail on the first pass. When that happens there is
> > a message that would look something like "redoing
> > filename(1524)" in stdout. It is possible you are
> > encountering this problem and -P is hiding it. Try
> > redirecting stdout, don't use -P and perhaps add an extra
> > -v.
> I don't remember seeing that error message. I tried your suggestion
> using this command:
> rsync -rtlvvessh /extra/library teep:/extra/tomg
> The standard output will be available here (warning: it's 3.1MiB):
404 Not Found
> The string "redoing" is not in the file of the standard output:
> tomg at nova:~/mylibrary$ grep redoing rsync_debug.0
> tomg at nova:~/mylibrary$
> I had run the touch command on the Knoppix ISO file like I did in the
> original post in this thread. Rsync is definitely uploading the Knoppix
> ISO again:
> -rw------- 1 tomg users 146636800 Apr 21 19:20 .KNOPPIX_V3.2-2003-04-18-EN.iso.FFb9TM
> -rw-r--r-- 1 tomg users 729716736 Apr 21 18:33 KNOPPIX_V3.2-2003-04-18-EN.iso
> So, are you convinced I have found a bug yet?
The existence of the temp file does not mean it is uploading
The way rsync updates a file is that it creates a temp file
and the reciever sends block checksums of the original to
the sender. The sender uses those blocksums to find the
parts of the file that have not changed and sends the parts
of the file that have changed and the offsets of unchanged
parts to the reciever which combines the original file and
the new data into the temp file. After the temp file is
done it replaces the original file with it.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync