Feature Request - Load Throttling

[jw schultz]

On Tue, Feb 17, 2004 at 07:34:06PM -0800, Marc Perkel wrote:
> 
> 
> jw schultz wrote:
> 
> >On Tue, Feb 17, 2004 at 06:47:19PM -0800, Marc Perkel wrote:
> > 
> >
> >>Actually - the problem is disk IO. And the disk IO is what makes the 
> >>load levels go up. The load level is something that's readable can can 
> >>be used to have rsync slow itself down. Nice doesn't do the trick. Nice 
> >>helps - but even at nice +19 it still slows the system to a crawl when 
> >>backing up from one drive to another.
> >>   
> >>
> >
> >Is that is on AIX with 12 AS400 CPUs or the VMS SSI cluster?
> >Or is that a single CPU linux box with a 2.4.?? kernel?
> > 
> >
> It's on a dual xeon with 4 gigs of ram and a pair of 250 gig serial ata 
> drives.
> 
> > 
> >
> >>So - if rsync could watch the load levels and pause every now and then 
> >>to put a little space between disk access at high load levels it would 
> >>make it a lot friendlier to the system. The reason nice doesn't work is 
> >>that once the system call is made to access the disk - nice doesn't apply.
> >>   
> >>
> >
> >What load levels?  Do you have some nice C code that can do
> >that for ALL the platforms without misreading?
> >
> >This is what process and i/o schedulers are for.
> >Maybe you should contact the people responsible for whatever
> >kernel it is you are running.
> >
> > 
> >
> What happens is that the server is cooking along just fine serving about 
> 2 million hits a day. Load level - according to top is running around 
> 0.6 to 2.3 or so - and then rsync kicks in doing a backup between the 
> drives and even though I'm running at nice +19 the load kicks up to 
> around 50 and several services almost stop. That's why I'm asking for 
> this feature.

By load do you mean that fuzzy approximation of the average
number of processes in the run queue.  I've been on systems
where a load average of 50 is normal and quite zippy.  Load
average is not a reliable number without context.

What you have is a failure of the scheduler.  The kernel
guys are working on that.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt