Rsync doesn't copy over only the changed files

[Chad]

Hello,

I am trying to sync up 2 directories over a network.  I am running
Gentoo and on the new server I have a fresh install of Apache using
webapp-config to install the latest Joomla 1.5.  On the other server I
have my backup copy of Joomla 1.5 also running the latest version.  I
want to only copy over the files that are not in my new "fresh"
version AND only the files that have different content in them so I
won't be overwriting the entire directory; only the changes I have
made.  The command I am using is:
rsync -avl -n --stats --progress
root at backup-server:/var/www/localhost/htdocs/Joomla Joomla

And I get all the files.  If I use -c --size-only or
--ignore-existing; nothing changes.  Every file will be transferred.
Am I missing something?  Does my problem make sense?

Thanks for any ideas!  I'm happy to provide more information if I'm
missing something above, just let me know.

-Chad