Parallel rsync's for better Performance.

[Satish Shukla]

Hi ,

We have huge data to sync usually everyday and I wish rsync could guarantee performance.

I thought of spliting the directories and run parallel rsyncs on them. It may cost me some network, but I can control that from the MAX_RSYNC_PROCESS variable. Can some one evaluate pros and cons of this design?. Any help is heartily appreciated. 



#!/usr/bin/ksh

MAX_RSYNC_PROCESS=10                           # Control the Parallelism from here

sync_and_wait()
{
  i=0
  while read RSYNC_COMMAND
  do
  eval "${RSYNC_COMMAND}" &                    # The command is rsync command line
  i=$((i+1))
  if [[ $i = ${MAX_RSYNC_PROCESS} ]]
  then
    wait
    i=0
  fi
  done
  wait
}




Thanks,
Satish Shukla