Question about rsync and BIG mirror

johan.boye at latecoere.fr johan.boye at latecoere.fr
Thu Mar 2 14:56:51 GMT 2006


Hello,

  I'm quite a n00b on rsync stuff but I went to the website, read
FAQ/how-to, Google and more, I setup my own rsync server and clients:
everything works fine :-D

  I'm preparing a plan for a production mode in my company: we need to
mirror around 100GB of data trough a special VPN internet line 2MB
symmetric.
  The first time, the data will be transferred by a media such as a HD.
Next, each night, we will try to update clients from the master server.
It should be around 500MB to 3GB, no so much in comparison of the
original size of data. 
  I discovered "rsync" use a lot of CPU and RAM to run "checksums" on
file that have to be synchronised. I need an opinion about my situation:


  So: each night, from 0:00am to maximum 7:00am, the server will have to
check the 100Go of files and see what files have been modified, then,
upload them to the clients. Each file is around 4MB to 40MB in average. 

I would like to know your opinion about this situation:  
 - Should I setup a strong dual CPU computer dedicated to calculate this
whole stuff? 
 - What about the memory I should install? 
 - Is there any bandwidth used during the checksums computation? Mine is
quite limited.
 - I know the client computer will have to check files too; Disk I/O
will be the most used. I think this computer will have NFS mount from a
"datacenter" computer with a GB LAN card, I wonder it will be enough...

  I'm quite scared of the amount of data to check before synchronise
clients, and how long it will take. To finish shortly, what do YOU
think? Any advices?


Thanks,

Johan


More information about the rsync mailing list