20 May
2021
20 May
'21
5:38 p.m.
Hello Members,
I have kamailio setup backed with mysql db, using db_cluster module for
spliting read and write..
Lisiting on three different virtual ips are bind with same/single udp port
*1* for processing calls.
One static IP is bind with udp port*2* to for processing/response to
option messages send from keepalived external script after every few
seconds and for DMQ messages from other kamailio node.
General configuation details:
children=16
fork=yes
listen=udp:VIP:SIP_PORT1
listen=udp:VIP2:SIP_PORT1
listen=udp:VIP3:SIP_PORT1
listen=udp:StaticIP3:SIP_PORT2
listen=tcp:SIP:RPC_PORT
SH_MEM=5012
PRIV_MEM=264
System Specification : 24 GB RAM, 12 CPUs
Sometimes kamailio radomly stop processing the requests or give no
response on new requests recevied on virtual ips and as well it did't show
any error in syslog.
To monitor the kamailio memory I used promethues (
https://github.com/florentchauveau/kamailio_exporter)
The moment kamailio stop processing the requests it also stop responding to
kamailio exporter and scrap request get timeout. i.e error log on console
debian prometheus-kamailio-exporter[83440]: 2021/05/20 12:15:27 [error]
cannot read header: read unix @->/tmp/kamailio__ctl: i/o timeout
After getting first timeout occurrence of kamailio-exporter scrap request,
kamailio processed sip messages for almost 30 seconds then completly stop
replying to any message. But it keep replying to option message arrive on
different port from keepalived script.
Stats :
Active dialog: 75
Shared memory used: 150 MB
Kamailio Installed version :
kamailio -v
version: kamailio 5.4.1 (x86_64/linux) 2ca223
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC,
TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024,
BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 2ca223
compiled on 12:51:24 Sep 21 2020 with gcc 6.3.0
Is anyone else face similar issue or idea why kamailio is showing such
behaviour.?
Please guide me on what I should do to find and fix the issue.
Thanks
--
Regards
M Arqum
21 May
21 May
10:42 a.m.
Hello,
just a quick note – 264 MB of private memory is a lot, as it is per process. Usually, you
do not need that much.
Regarding the error/hang of Kamailio, do you see any errors in the log file, e.g., an out
of memory error or similar?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com<https://gilawa.com/>
From: sr-users <sr-users-bounces(a)lists.kamailio.org> On Behalf Of M Arqum CH
Sent: Thursday, May 20, 2021 4:38 PM
To: Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>
Subject: [SR-Users] Kamailio Stop Processing Call
Hello Members,
I have kamailio setup backed with mysql db, using db_cluster module for spliting read and
write..
Lisiting on three different virtual ips are bind with same/single udp port1 for
processing calls.
One static IP is bind with udp port2 to for processing/response to option messages
send from keepalived external script after every few seconds and for DMQ messages from
other kamailio node.
General configuation details:
children=16
fork=yes
listen=udp:VIP:SIP_PORT1
listen=udp:VIP2:SIP_PORT1
listen=udp:VIP3:SIP_PORT1
listen=udp:StaticIP3:SIP_PORT2
listen=tcp:SIP:RPC_PORT
SH_MEM=5012
PRIV_MEM=264
System Specification : 24 GB RAM, 12 CPUs
Sometimes kamailio radomly stop processing the requests or give no response on new
requests recevied on virtual ips and as well it did't show any error in syslog.
To monitor the kamailio memory I used promethues
(https://github.com/florentchauveau/kamailio_exporter)
The moment kamailio stop processing the requests it also stop responding to kamailio
exporter and scrap request get timeout. i.e error log on console
debian prometheus-kamailio-exporter[83440]: 2021/05/20 12:15:27 [error] cannot read
header: read unix @->/tmp/kamailio__ctl: i/o timeout
After getting first timeout occurrence of kamailio-exporter scrap request, kamailio
processed sip messages for almost 30 seconds then completly stop replying to any message.
But it keep replying to option message arrive on different port from keepalived script.
Stats :
Active dialog: 75
Shared memory used: 150 MB
Kamailio Installed version :
kamailio -v
version: kamailio 5.4.1 (x86_64/linux) 2ca223
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST,
DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY,
USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR,
USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535,
DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 2ca223
compiled on 12:51:24 Sep 21 2020 with gcc 6.3.0
Is anyone else face similar issue or idea why kamailio is showing such behaviour.?
Please guide me on what I should do to find and fix the issue.
Thanks
--
Regards
M Arqum
12:16 p.m.
Hi Henning,
Thanks for reply and information regarding memory.
No error log in log file regarding anything.
Prometheus scrapper just shows as shared unable to connect with kamailio
socket.
Thanks
Arqum
On Fri, May 21, 2021, 12:42 PM Henning Westerholt <hw(a)skalatan.de> wrote:
Hello,
just a quick note – 264 MB of private memory is a lot, as it is per
process. Usually, you do not need that much.
Regarding the error/hang of Kamailio, do you see any errors in the log
file, e.g., an out of memory error or similar?
Cheers,
Henning
--
Henning Westerholt – https://skalatan.de/blog/
Kamailio services – https://gilawa.com
*From:* sr-users <sr-users-bounces(a)lists.kamailio.org> *On Behalf Of *M
Arqum CH
*Sent:* Thursday, May 20, 2021 4:38 PM
*To:* Kamailio (SER) - Users Mailing List <sr-users(a)lists.kamailio.org>
*Subject:* [SR-Users] Kamailio Stop Processing Call
Hello Members,
I have kamailio setup backed with mysql db, using db_cluster module for
spliting read and write..
Lisiting on three different virtual ips are bind with same/single udp
port*1* for processing calls.
One static IP is bind with udp port*2* to for processing/response to
option messages send from keepalived external script after every few
seconds and for DMQ messages from other kamailio node.
General configuation details:
children=16
fork=yes
listen=udp:VIP:SIP_PORT1
listen=udp:VIP2:SIP_PORT1
listen=udp:VIP3:SIP_PORT1
listen=udp:StaticIP3:SIP_PORT2
listen=tcp:SIP:RPC_PORT
SH_MEM=5012
PRIV_MEM=264
System Specification : 24 GB RAM, 12 CPUs
Sometimes kamailio radomly stop processing the requests or give no
response on new requests recevied on virtual ips and as well it did't show
any error in syslog.
To monitor the kamailio memory I used promethues (
https://github.com/florentchauveau/kamailio_exporter)
The moment kamailio stop processing the requests it also stop responding
to kamailio exporter and scrap request get timeout. i.e error log on
console
debian prometheus-kamailio-exporter[83440]: 2021/05/20 12:15:27 [error]
cannot read header: read unix @->/tmp/kamailio__ctl: i/o timeout
After getting first timeout occurrence of kamailio-exporter scrap
request, kamailio processed sip messages for almost 30 seconds then
completly stop replying to any message. But it keep replying to option
message arrive on different port from keepalived script.
Stats :
Active dialog: 75
Shared memory used: 150 MB
Kamailio Installed version :
kamailio -v
version: kamailio 5.4.1 (x86_64/linux) 2ca223
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC,
F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT,
USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024,
BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 2ca223
compiled on 12:51:24 Sep 21 2020 with gcc 6.3.0
Is anyone else face similar issue or idea why kamailio is showing such
behaviour.?
Please guide me on what I should do to find and fix the issue.
Thanks
--
Regards
M Arqum
1284
days inactive
1285
days old
2 comments
2 participants
participants (2)
-
Henning Westerholt -
M Arqum CH