[clug] Out of Memory: Kill process 2689 (mysqld) score 33827 and children.

[Daniel Pittman]

David Schoen <dave at lyte.id.au> writes:
> On 18 August 2010 12:23, Daniel Pittman <daniel at rimspace.net> wrote:
>
>>> Tuning is always a good thing, but if it's a deadlock issue it will not
>>> solve your current issue.
>>
>> Yeah, I just have trouble believing that there is anything particularly
>> likely to involve a deadlock in this scenario.
>>
>> Actually, I am surprised you see so many deadlocks in MySQL, frankly.  Pretty
>> much none of the PHP applications I am familiar with do any locking at all,
>> just random updates without transactions.  Which packages usually trigger this
>> sort of lock problem?
>
> I'm much more familiar with one specific application - MySource Matrix. It
> runs on PostgreSQL and Oracle, not MySQL. Specifc problems vary drastically
> but one great way to deadlock PHP is to make use of the native session
> handler and find any old URL that crashes that thread, then just keep
> sending in requests until Apache runs out of slots (or in this case gets
> killed off by OOM).

Ouch.  Forgive me for having to say it, but this is a serious question: is
this something you found across multiple PHP versions, or was it limited to a
particular buggy release or something?

I don't manage enough PHP (increasingly thankfully) that I have run into this
much, so would have imagined that Apache would relatively gracefully handle
this, by eventually terminating the worker at the very least...

[...]

>> *nod*  Very strange.  It sounds like a whole lot of the problems come from the
>> range of crappy PHP "accelerators" that are intended to work around the nasty,
>> non-persistent nature of PHP.  Is that a fair summary?
>
> A lot of issues are related to that. We've had a lot of interaction problems
> that "no one else sees" with all sorts of PHP accelerators.

We have a developer who I just love to point at new services: he has the magic
touch that brings those out.  Someone like that is great to have around,
especially for me: I seem to have a magic touch of "works fine" for a whole
lot of these things, so I just suck at finding these production bugs. ;)

[...]

> The point is still to strace/lsof the procs and see what's going on, to do
> that the kernel needs to remain responsive while Apache crashes, which
> usually means lowering MaxClients.

*nod* I confess, one reason I prefer FastCGI (style) operation for PHP (and
most other languages) is that it isolates the web server and application
server life-cycle.

Not always perfectly, but better than the in-process handlers.  That usually
makes it easier to tune, and often easier to support more client concurrency,
than doing it all in one big mess.

OTOH, I also prefer not to use Apache on the front end anyway, so my tastes
are clearly outside the routine. ;)

        Daniel

Nginx, if you really want to know.
-- 
✣ Daniel Pittman            ✉ daniel at rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons