28 jul 2010 kl. 17.29 skrev Iñaki Baz Castillo:
Hi, finally I've discovered the cause of important issues with my Kamailio deployments which sometimes get frozen "with no reason":
- We use Debian Lenny with rsyslog writting the logs in local files
(/var/log/kamailio.log) and also rsyslog-mysql extension to send the logs to a remote MySQL server using mysql TCP protocol.
- For "external" reason (external to me) such MySQL server sometimes
fails or it's down.
- Then kamailio tries to log to syslog. rsyslog receives the request
and tries to send it via MySQL. Unfortunatelly it gets frozen as the DB doesn't work, and it neither writes the log to the local log file !!! (this is really surprising to me).
- Worse is the fact the rsyslog replies nothing to kamailio process so
the kamailio process gets 100% *blocked* waiting for the response (this is what I suspect).
- As a dirty workaround, stopping the local rsyslog server makes
kamailio to work again.
So, I think this is mostly a bug in rsyslog-mysql as it shouldn't get frozen if the MySQL server doesn't work. However I also think that Kamailio should not depend on it, and perhaps it shouldn't wait indefinitely for the rsyslog response. What do you think? Would make sense a "ping" mechanism like with RtpProxy servers in order to enable/dissable the syslog logging?
Syslog is defined in http://tools.ietf.org/html/rfc5424, but I can't find anything there about this situation. It's a simplex protocol, so the implementation should propably handle this situation, as you say, rsyslog is at fault here.
If it does, we need to figure out a way so that Kamailio doesn't lock up if this happens, not logging in the same process or thread or something...
/O