Hi folk! Have a production server - used as webrtc to sip proxy - it is built on kamailio v 4.4.2 - Server handles connections from ~4000 incoming SIP trunks (transport=TLS). Kamailio provides auth for this trunks. - Kamailio receives also connections from Webrtc app. Calls from Webrtc routed to one of the 4000 sip trunks.
Server is running on Debian GNU/Linux 7
The number of calls on the system is small. Most of the load is generated by registration/re-registrations of SIP trunks
The problem is following: - output of kamcmd core.shmmem and kamcmd pkg.stats shows constant using of memory - output of htop shows constantly increasing memory usage, e.g. after one week working: PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command ..... 6013 kamailio 20 0 744M 134M 130M S 0.0 3.6 2:28.80 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 32 -u kamailio -g kamailio 6002 kamailio 20 0 744M 134M 130M S 0.0 3.6 2:30.93 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 32 -u kamailio -g kamailio 6012 kamailio 20 0 744M 134M 130M S 0.0 3.5 2:35.00 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 32 -u kamailio -g kamailio 5991 kamailio 20 0 744M 133M 129M S 0.0 3.5 2:05.88 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 32 -u kamailio -g kamailio 6018 kamailio 20 0 744M 133M 129M S 0.0 3.5 2:31.90 /usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.pid -m 512 -M 32 -u kamailio -g kamailio .....
Despite running with: PKG_MEMORY=32, Amount of used memory per process is higher. Finishes all with monit killing kamailio, because of memory usage more than allowed
studied this tutorial: https://www.kamailio.org/wiki/tutorials/troubleshooting/memory but did not find a way how to dig up the problem's root.
Tried also to reproduce the issue in lab environment (having clone of production server). Generated load with incoming registration using PJSIP, but with no success - memory usage is constant.
kamailio -v version: kamailio 4.4.2 (x86_64/linux) flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 4.7.2
Any suggestion/idea how to troubleshoot deeper would be highly apprecieted
Cheers!
-- View this message in context: http://sip-router.1086192.n5.nabble.com/memory-is-not-freed-in-system-while-... Sent from the Users mailing list archive at Nabble.com.