full upload happening even though only a timestamp has changed

[Donovan Baarda]

On Tue, 2003-04-22 at 11:32, Tom Goulet wrote:
> > > As you can see, the second use of Rsync wrote 729805852 more bytes than
> > > the first.
> > The first time it didn't update the iso file.
> 
> I realise that... My problem is that it shouldn't have updated the file
> the second time, since the remote file was identical to the local file.

The second time the file had a different timestamp, so it "rsync'ed" it.
This means it rebuilt the file using the existing file as a basis, using
the rsync algorithm to try and find "matching" blocks in the existing
file. The fact that they all matched means it didn't need to transfer
any data, but it still needed to negotiate with upstream to determine
that they "matched".

> Okay, I got all that.  So if the modification time is different, Rsync
> should get a checksum, right?  And if the checksum is the same, Rsync
> should not upload, right?  But in this case, Rsync is uploading even
> though the checksum should be is the same.

The checksum bit as you describe it only occurs with the '-c' option.
Without this option rsync generates a "signature" that is transmitted
upstream and used to generate a "delta" that is applied to the "basis"
to update the file. If the files turn out to be identical the delta
turns out to be very small (ie, "nothing changed"), but it is still
applied to update the file.

> > There is a known problem with files of this size where block
> > checksums fail on the first pass.  When that happens there is
> > a message that would look something like "redoing
> > filename(1524)" in stdout.  It is possible you are
> > encountering this problem and -P is hiding it.  Try
[...]
> I had run the touch command on the Knoppix ISO file like I did in the
> original post in this thread.  Rsync is definitely uploading the Knoppix
> ISO again:
> -rw-------    1 tomg     users    146636800 Apr 21 19:20 .KNOPPIX_V3.2-2003-04-18-EN.iso.FFb9TM
> -rw-r--r--    1 tomg     users    729716736 Apr 21 18:33 KNOPPIX_V3.2-2003-04-18-EN.iso
> 
> So, are you convinced I have found a bug yet?

No... this does not mean it is "downloading" it again, just that it is
"rsyncing it again". Usually this means it is just getting a list of
"block not changed" commands from upstream and copying the file. 

However, from the "wrote 731164664 bytes" it looks like you might have
hit the known problem jw mentioned. The workaround for this is to use a
sufficiently large block size. Try using -B 65536.

BTW, how fast is the connection? 700M in 11 minutes is pretty fast. 10M
ethernet?

-- 
----------------------------------------------------------------
Donovan Baarda                http://minkirri.apana.org.au/~abo/
----------------------------------------------------------------