High CPU on multiple syncs on Win2K
jw at pegasys.ws
Fri Aug 23 11:10:00 EST 2002
On Fri, Aug 23, 2002 at 09:39:03PM +0200, bart.coninckx at watco.be wrote:
> >Some things that would help in evaluation would be the
> >rsync command-line and the disk configuration (are you running
> >RAID, what level and HW vs. SW, etc), approximate file counts
> >and size, and it also might help if we knew if the overloaded
> >server were low powered.
> this is the command line of the server:
> rsync --daemon --config=rsyncd.conf
> rsync -rtv --delete --modify-window=2 --stats /cygdrive/r/
> We're using hardware RAID 5.
> >As a start i would check to be sure that you aren't using
> >the --checksum option unnecessarily.
> Nope, we use the default...
> >That was apparently the
> >problem the last time someone reported excessive load.
> >Also, don't run software RAID-5.
> No, we indeed prefer hardware RAID. We use a SCSI to ATA RAID configuration
> (a device with a SCSI connector, but with IDE disks in it), but this should
> not influence CPU performance, since the machine sees natice SCSI.
OK we've covered the obvious. I only mentioned SW RAID
because that can turn IO bound to CPU bound.
A server change might affect the number of client
connections that are supportable. If you are CPU bound,
changing hardware will probably have more effect than
> >The other thing that would be worth doing is to manage the
> >load better by reducing the number of simultaneous
> True, but we have a limited timeframe (about 10 hours), to sync everything.
> It could well be that Rsync manages to do an avarage site in less than 2
> hours, but we don't know yet. There are a lot of variables there: what is
> changed in data on a particular day on a particular site, how fast is the
> WAN link, ... Since the clients are scattered, it's also hard to
> orchestrate a steady scheme. This would have been a lot easier in a pull
> configuration, but as I mentionned, for some reason the Rsync service
> (daemon) won't work with Netware mappings.
Look into the rsyncd.conf "max connections" parameter. I
haven't used it myself but expect that it will require
wrapping the job in a script to do a
collision_detect+backoff+retry loop. That would allow you
to have much less wasted window-time than you would with
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync