rsycnc copies all files

[Tim Conway]

Use "-a" unless there's a reason you need to retain different permissions, 
owners, etc..
"-c" is pretty much for those situations wherein you end up with identical 
times and size but different content, and you need rsync to look inside 
even though it appears unnecessary.  As you have seen, slower than heck.

I don't know the nature of your filesystems, but I have a guess... on at 
least one end is a network filesystem - NFS, SMB, NCP, AFS, something like 
that.  rsync has the "-W", or "--whole-file" option, which tells it that 
there's no point in trying to read the file for changes - if it needs 
transferring, just send the whole thing, because the LAN overhead will 
waste the savings in WAN.  Last I heard, -W was going to be forced if 
either end was a network filesystem.

Wayne:  did that ever come to pass?

Another thing:  I haven't seen your actual commandlines and results, just 
descriptions.  Here's a set of sample runs.  Note the difference between 
"total size" and "wrote", and the fact that 29 or so bytes + 73 bytes nop 
overhead doesn't come close to 470, or 6806.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
cnwt99 at chaos
/home/cnwt99/work>sh doit 
mkdir source
ssh disaster mkdir destination
dd if=/usr/local/admin/lib/locatedb of=source/Kfile bs=1024 count=1
1+0 records in
1+0 records out
rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile 
disaster:destination
building file list ... done
Kfile
wrote 1137 bytes  read 36 bytes  782.00 bytes/sec
total size is 1024  speedup is 0.87
rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile 
disaster:destination
building file list ... done
wrote 73 bytes  read 20 bytes  62.00 bytes/sec
total size is 1024  speedup is 11.10
date >>source/Kfile
rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile 
disaster:destination
building file list ... done
Kfile
wrote 470 bytes  read 48 bytes  345.33 bytes/sec
total size is 1053  speedup is 2.30
rsync -av --rsync-path=/usr/local/bin//rsync source/Kfile 
disaster:destination
building file list ... done
wrote 73 bytes  read 20 bytes  62.00 bytes/sec
total size is 1053  speedup is 11.32
dd if=/usr/local/admin/lib/locatedb of=source/Mfile bs=1024 count=1024
1024+0 records in
1024+0 records out
rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile 
disaster:destination
building file list ... done
Mfile
wrote 1048813 bytes  read 36 bytes  699232.67 bytes/sec
total size is 1048576  speedup is 1.00
rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile 
disaster:destination
building file list ... done
wrote 73 bytes  read 20 bytes  62.00 bytes/sec
total size is 1048576  speedup is 11275.10
date >>source/Mfile
rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile 
disaster:destination
building file list ... done
Mfile
wrote 6806 bytes  read 9024 bytes  6332.00 bytes/sec
total size is 1048605  speedup is 66.24
rsync -av --rsync-path=/usr/local/bin//rsync source/Mfile 
disaster:destination
building file list ... done
wrote 73 bytes  read 20 bytes  62.00 bytes/sec
total size is 1048605  speedup is 11275.32
cnwt99 at chaos
/home/cnwt99/work>date
Thu Jun 17 16:11:23 MDT 2004
cnwt99 at chaos
/home/cnwt99/work>date |wc -c
      29
cnwt99 at chaos
/home/cnwt99/work>
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Note also that the delta transfer doesn't happen on local transfers. Since 
to do so would be to build a new copy of the file at the destination, 
reading from both the source and destination files (once in entirety for 
the checksums, again partially for the pieces kept and sent), then 
deleting the destination, then renaming the newly created file to the 
destination, it just does it in one read/write/unlink/rename... 
essentially like the "-W" option.

Good luck.

Tim Conway
Unix System Administration
Contractor - IBM Global Services
desk:3032734776
conway at us.ibm.com




Wayne Davison wrote:

>On Wed, Jun 16, 2004 at 08:26:33PM +0100, Gareth wrote:
> 
>
>>I am making an extraordinary claim: rysnc seems to copy all my files, 
>>not just ones that have changed or new files.
>> 
>>
>
>Use either -t (preferred) or -c (slower).  See also -a.
>
>..wayne..
> 
>

Using -a, -t or -c (vvv slow!) does resolve the problem, but now I 
notice that appends to text files results in the whole file being 
transferred (as recorded by 'Total bytes written') rather than just the 
9 characters I append to the file.

Surely this isn't the desired outcome?

Any information much appreciated.


Gareth