Feature Request - Load Throttling
jw at pegasys.ws
Wed Feb 18 03:42:04 GMT 2004
On Tue, Feb 17, 2004 at 07:34:06PM -0800, Marc Perkel wrote:
> jw schultz wrote:
> >On Tue, Feb 17, 2004 at 06:47:19PM -0800, Marc Perkel wrote:
> >>Actually - the problem is disk IO. And the disk IO is what makes the
> >>load levels go up. The load level is something that's readable can can
> >>be used to have rsync slow itself down. Nice doesn't do the trick. Nice
> >>helps - but even at nice +19 it still slows the system to a crawl when
> >>backing up from one drive to another.
> >Is that is on AIX with 12 AS400 CPUs or the VMS SSI cluster?
> >Or is that a single CPU linux box with a 2.4.?? kernel?
> It's on a dual xeon with 4 gigs of ram and a pair of 250 gig serial ata
> >>So - if rsync could watch the load levels and pause every now and then
> >>to put a little space between disk access at high load levels it would
> >>make it a lot friendlier to the system. The reason nice doesn't work is
> >>that once the system call is made to access the disk - nice doesn't apply.
> >What load levels? Do you have some nice C code that can do
> >that for ALL the platforms without misreading?
> >This is what process and i/o schedulers are for.
> >Maybe you should contact the people responsible for whatever
> >kernel it is you are running.
> What happens is that the server is cooking along just fine serving about
> 2 million hits a day. Load level - according to top is running around
> 0.6 to 2.3 or so - and then rsync kicks in doing a backup between the
> drives and even though I'm running at nice +19 the load kicks up to
> around 50 and several services almost stop. That's why I'm asking for
> this feature.
By load do you mean that fuzzy approximation of the average
number of processes in the run queue. I've been on systems
where a load average of 50 is normal and quite zippy. Load
average is not a reliable number without context.
What you have is a failure of the scheduler. The kernel
guys are working on that.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync