rsync and many files

Cliff Simon cliff.simon at netclusive.com
Mon Jun 6 06:04:13 MDT 2011


Hello together,

I have a question about using rsync with many files.

We are using rsync via rsnapshot, but this is not elementary. It is used to backup many (above 100 servers) and works very well. Now there is one server with many (several millions) files. The files are not very big, so the complete backup is about 500 GB.

Now my problem is, that the backup needs about 14 hours - the most time is to generate the filelist and check whether the files are new/changed or not.

My rsync-command is:
/usr/bin/rsync -a --bwlimit=9000 --delete --numeric-ids --relative --delete-excluded --exclude=/some/pathes/ --rsh=/usr/bin/ssh --link-dest=/dest.path/daily.1/ root at 192.x.x.x:/path.to.backup/

Do you have an idea to reduce the backup time?

Btw: The bwlimit should not be the problem, because generating the filelist is the most time.

Thank you very much!
Cliff Simon



More information about the rsync mailing list