bug report: different files are not rsynced, identical files rsynced

Peter Chun peterch at ticketek.com.au
Thu Jul 24 09:52:44 EST 2003


Hi,
I have only just subscribe to the list.  ( only to send this bug report )

Running rsync  version 2.5.6  protocol version 26 ( on Solaris 8 sparc ) 
on both hosts.

I have 1 file I wish to sync to a remote machine
the md5 checksum is

host1: MD5 (030722.mj) = 020397fde83c2e20464b6642c018ce6e
host2: MD5 (030722.mj) = 35fcffc896c65c8ec861385b4edb81ac

For brevity ..
set RSYNC="--archive --verbose --stats --recursive --times --perms 
--owner --group --links --rsync-path=/usr/local/bin/rsync"

rsync $RSYNC --dry-run /path/030722.mj host2:/path/030722.mj
building file list ... done

Number of files: 1
Number of files transferred: 0
Total file size: 9416161 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 55
Total bytes written: 67
Total bytes read: 20

wrote 67 bytes  read 20 bytes  58.00 bytes/sec
total size is 9416161  speedup is 108231.74

The file is not listed.

There are other files in that path ( 030701.mj to 030722.mj )
All files from 030701 to 030721 have identical md5 checksums

rsync $RSYNC --dry-run /path/ host2:/path/
building file list ... done

Number of files: 23
Number of files transferred: 0
Total file size: 356814662 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 387
Total bytes written: 399
Total bytes read: 20

wrote 399 bytes  read 20 bytes  838.00 bytes/sec
total size is 356814662  speedup is 851586.31

I then copy the entire directory to /tmp/path/
rsync $RSYNC --dry-run /path/ host2:/tmp/path/
building file list ... done

Number of files: 23
Number of files transferred: 0
Total file size: 356814662 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 387
Total bytes written: 399
Total bytes read: 20

wrote 399 bytes  read 20 bytes  279.33 bytes/sec
total size is 356814662  speedup is 851586.31


If I make a copy of the file on host1 to /tmp


rsync $RSYNC --dry-run /tmp/030722.mj host2:/path/030722.mj
building file list ... done
030722.mj

Number of files: 1
Number of files transferred: 1
Total file size: 9416161 bytes
Total transferred file size: 9416161 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 50
Total bytes written: 66
Total bytes read: 24

wrote 66 bytes  read 24 bytes  60.00 bytes/sec
total size is 9416161  speedup is 104624.10

MD5 (/path/030722.mj) = 020397fde83c2e20464b6642c018ce6e
MD5 (/tmp/030722.mj) = 020397fde83c2e20464b6642c018ce6e


Also I have copied the directory to host1 ( remembering that all 030701 
to 030721 files have identical md5 checksums )
rsync $RSYNC --dry-run /path/0307/ /path2/0307/        
building file list ... done
030701.mj
030702.mj
030703.mj
030704.mj
030705.mj
030706.mj
030707.mj
030708.mj
030709.mj
030710.mj
030711.mj
030712.mj
030713.mj
030714.mj
030715.mj
030716.mj
030717.mj
030718.mj
030719.mj
030720.mj
030721.mj
030722.mj

Number of files: 23
Number of files transferred: 22
Total file size: 356814662 bytes
Total transferred file size: 356814662 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 387
Total bytes written: 487
Total bytes read: 108

wrote 487 bytes  read 108 bytes  1190.00 bytes/sec
total size is 356814662  speedup is 599688.51

Note: there are 23 files in the dir, a --delete will remove 030723.mj

The --check-sum flag is ( as stated in the docs ) extremely slow.
rsyncing without already takes about 3hrs
timing tests show that it could take upto 3600 hours to complete with 
--check-sum
These files have cross references, thus you can not just rsync todays file.
Thus the need to rsync the entire path.

Thanks
Peter







More information about the rsync mailing list