High CPU on multiple syncs on Win2K

[jw schultz]

On Fri, Aug 23, 2002 at 09:39:03PM +0200, bart.coninckx at watco.be wrote:
> >Some things that would help in evaluation would be the
> >rsync command-line and the disk configuration (are you running
> >RAID, what level and HW vs. SW, etc), approximate file counts
> >and size, and it also might help if we knew if the overloaded
> >server were low powered.
> 
> this is the command line of the server:
> rsync --daemon --config=rsyncd.conf
> 
> client:
> rsync -rtv --delete --modify-window=2 --stats /cygdrive/r/
> bee2bs01::d/backup/servername/data/
> 
> We're using hardware RAID 5.
> 
> >As a start i would check to be sure that you aren't using
> >the --checksum option unnecessarily.
> 
> Nope, we use the default...
> 
> >That was apparently the
> >problem the last time someone reported excessive load.
> >Also, don't run software RAID-5.
> 
> No, we indeed prefer hardware RAID. We use a SCSI to ATA RAID configuration
> (a device with a SCSI connector, but with IDE disks in it), but this should
> not influence CPU performance, since the machine sees natice SCSI.

OK we've covered the obvious.  I only mentioned SW RAID
because that can turn IO bound to CPU bound.

A server change might affect the number of client
connections that are supportable.  If you are CPU bound,
changing hardware will probably have more effect than
the OS.

> >The other thing that would be worth doing is to manage the
> >load better by reducing the number of simultaneous
> >connections.
> 
> True, but we have a limited timeframe (about 10 hours), to sync everything.
> It could well be that Rsync manages to do an avarage site in less than 2
> hours, but we don't know yet. There are a lot of variables there: what is
> changed in data on a particular day on a particular site, how fast is the
> WAN link, ... Since the clients are scattered, it's also hard to
> orchestrate a steady scheme. This would have been a lot easier in a pull
> configuration, but as I mentionned, for some reason the Rsync service
> (daemon) won't work with Netware mappings.

Look into the rsyncd.conf "max connections" parameter.  I
haven't used it myself but expect that it will require
wrapping the job in a script to do a
collision_detect+backoff+retry loop.  That would allow you
to have much less wasted window-time than you would with
pure scheduling.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt