Skip based on checksum not worked as expected when using with complex filter rules.

[Hongyi Zhao]

Hi all,

I've a local repository for Debian, and I'm doing some rsync testings on 
it.  See the following commands for detail:

werner at debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -n \ 
ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
debian-installer/binary-amd64/Packages.gz \
repo.debian.local/debian/
[snip]
dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz

sent 46 bytes  received 202 bytes  55.11 bytes/sec
total size is 20  speedup is 0.08 (DRY RUN)

werner at debian:~/software/LocalRepo$ rsync -amvHKLPRSB131072 -nc \ 
ftp.uk.debian.org::debian/./dists/wheezy-updates/main/\
debian-installer/binary-amd64/Packages.gz \
repo.debian.local/debian/
[snip]
sent 46 bytes  received 218 bytes  58.67 bytes/sec
total size is 20  speedup is 0.08 (DRY RUN)

As you can see, if I use checksum on that package, it shown that it's 
just the same to my local copy, so there is not need to be rsynced.

But when I use a complex rsync filter rules script based on checksum and 
others to check all of the stuff based on the corresponding rules, it 
shows that the above package need to be updated, see the following for 
detail:

1- My script is named as su_dists.rsync and the content of it is as 
follows, sorry for its complex ;-( 

___________ begin su_dists.rsync ____________

rsync -amvHKLPRSB131072 -nc --delete --delete-excluded \
-f +_dists/wheezy/**Translation-en* \
-f +_dists/wheezy/**binary-all/Packages.gz \
-f +_dists/wheezy/Release* \
-f P_dists/wheezy-backports/**Translation-en* \
-f P_dists/wheezy-backports/**binary-all/Packages.gz \
-f P_dists/wheezy-backports/Release* \
-f P_dists/wheezy-proposed-updates/**Translation-en* \
-f P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \
-f P_dists/wheezy-proposed-updates/Release* \
-f +_dists/wheezy-updates/**Translation-en* \
-f +_dists/wheezy-updates/**binary-all/Packages.gz \
-f +_dists/wheezy-updates/Release* \
-f +_dists/wheezy/**binary-amd64/Packages.gz \
-f +_dists/wheezy/**installer-amd64/current/*** \
-f P_dists/wheezy-backports/**binary-amd64/Packages.gz \
-f P_dists/wheezy-backports/**installer-amd64/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \
-f +_dists/wheezy-updates/**binary-amd64/Packages.gz \
-f +_dists/wheezy-updates/**installer-amd64/current/*** \
-f +_dists/wheezy/**binary-i386/Packages.gz \
-f +_dists/wheezy/**installer-i386/current/*** \
-f P_dists/wheezy-backports/**binary-i386/Packages.gz \
-f P_dists/wheezy-backports/**installer-i386/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-i386/current/*** \
-f +_dists/wheezy-updates/**binary-i386/Packages.gz \
-f +_dists/wheezy-updates/**installer-i386/current/*** \
-f P_dists/jessie/*** \
-f P_dists/jessie-backports/*** \
-f P_dists/jessie-proposed-updates/*** \
-f P_dists/jessie-updates/*** \
-f P_pool/*** \
-f +_dists/***/ \
-f -_* ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/

___________ end su_dists.rsync ____________

2- Then, I run my above script as follows to obtain all of the files 
filtered-out according to my filter rules and checkout the ones need to 
be updated based on checksum:  

werner at debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/' \
'/^dists/ && $NF'
dists/wheezy-updates/Release
dists/wheezy-updates/Release.gpg
dists/wheezy-updates/contrib/binary-i386/Packages.gz
dists/wheezy-updates/contrib/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/contrib/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/main/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/non-free/binary-amd64/Packages.gz
dists/wheezy-updates/non-free/binary-i386/Packages.gz
dists/wheezy-updates/non-free/debian-installer/binary-amd64/Packages.gz
dists/wheezy-updates/non-free/debian-installer/binary-i386/Packages.gz
dists/wheezy-updates/non-free/i18n/Translation-en.bz2
dists/wheezy/contrib/debian-installer/binary-all/Packages.gz
dists/wheezy/contrib/debian-installer/binary-amd64/Packages.gz
dists/wheezy/contrib/debian-installer/binary-i386/Packages.gz
dists/wheezy/non-free/debian-installer/binary-all/Packages.gz
dists/wheezy/non-free/debian-installer/binary-amd64/Packages.gz
dists/wheezy/non-free/debian-installer/binary-i386/Packages.gz

As you can see, this time, the following file is considered to be updated:

dists/wheezy-updates/main/debian-installer/binary-amd64/Packages.gz

But it has been considered not need to be updated in my first run with 
the -c/--checksum option from the terminal other than the bash srcipt.

3- If I change my bash script into the following form, i.e., with the `--
no-t' option added to the args list:

___________ begin su_dists.rsync ____________

rsync -amvHKLPRSB131072 -nc --no-t --delete --delete-excluded \
-f +_dists/wheezy/**Translation-en* \
-f +_dists/wheezy/**binary-all/Packages.gz \
-f +_dists/wheezy/Release* \
-f P_dists/wheezy-backports/**Translation-en* \
-f P_dists/wheezy-backports/**binary-all/Packages.gz \
-f P_dists/wheezy-backports/Release* \
-f P_dists/wheezy-proposed-updates/**Translation-en* \
-f P_dists/wheezy-proposed-updates/**binary-all/Packages.gz \
-f P_dists/wheezy-proposed-updates/Release* \
-f +_dists/wheezy-updates/**Translation-en* \
-f +_dists/wheezy-updates/**binary-all/Packages.gz \
-f +_dists/wheezy-updates/Release* \
-f +_dists/wheezy/**binary-amd64/Packages.gz \
-f +_dists/wheezy/**installer-amd64/current/*** \
-f P_dists/wheezy-backports/**binary-amd64/Packages.gz \
-f P_dists/wheezy-backports/**installer-amd64/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-amd64/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-amd64/current/*** \
-f +_dists/wheezy-updates/**binary-amd64/Packages.gz \
-f +_dists/wheezy-updates/**installer-amd64/current/*** \
-f +_dists/wheezy/**binary-i386/Packages.gz \
-f +_dists/wheezy/**installer-i386/current/*** \
-f P_dists/wheezy-backports/**binary-i386/Packages.gz \
-f P_dists/wheezy-backports/**installer-i386/current/*** \
-f P_dists/wheezy-proposed-updates/**binary-i386/Packages.gz \
-f P_dists/wheezy-proposed-updates/**installer-i386/current/*** \
-f +_dists/wheezy-updates/**binary-i386/Packages.gz \
-f +_dists/wheezy-updates/**installer-i386/current/*** \
-f P_dists/jessie/*** \
-f P_dists/jessie-backports/*** \
-f P_dists/jessie-proposed-updates/*** \
-f P_dists/jessie-updates/*** \
-f P_pool/*** \
-f +_dists/***/ \
-f -_* ftp.uk.debian.org::debian/./ ./repo.debian.local/debian/

___________ end su_dists.rsync ____________

This time, I will obtain the correct file-lists that should be updated 
based on the checksum checking, see following for detail:

werner at debian:~/software/LocalRepo$ bash su_dists.rsync|awk -F'/' \
'/^dists/ && $NF'
dists/wheezy-updates/Release
dists/wheezy-updates/Release.gpg
dists/wheezy-updates/non-free/i18n/Translation-en.bz2

As you can see, for my case, there only have three files which need to be 
updated all toghether at the moment.

Any hints on the above issues?

Regards
-- 
.: Hongyi Zhao [ hongyi.zhao AT gmail.com ] Free as in Freedom :.