John Petrini writes:
The times we've seen this is when transactions are
waiting on something so
they pile up consuming shared memory. Do you have any database lookups or
calls out to external services or scripts?
John,
Thanks for your reply. Yes, there are MySQL operations both during
request/reply processing and also accounting to db. I would assume that
any problems with those should show up in /var/log/mysql/error.log.
Long timeouts can also contribute if something stops
responding because
transactions are waiting for a long timeout to expire.
There is quite long "fr_inv_timer" value, that could cause problems is
lots of INVITEs are ringing at the same time.
It's typically a balance between setting
reasonable timeouts and allocating
enough shm. In addition we implemented some watcher scripts that monitor
shm and will set gflags to disable non-critical external calls beyond a
certain threshold as well as send us an alert.
I also created an external script to watch shm memory usage. I would
have liked K to watch itself, but could not find pseudo variables
corresponding to core.shmmem entries. Do they exist?
-- Juha