sr-dev April 2025

sr-dev@lists.kamailio.org

14 participants
205 discussions

[kamailio/kamailio] prefix_route - prefix longer than 12 characters ignored (Issue #4218)
by Barry Flanagan 21 Jun '25

21 Jun '25

flantel created an issue (kamailio/kamailio#4218) ### Description If I add a prefix of greater than 12 characters in to prefix_route it does not return the correct route but instead acts as if no record was found. ### Troubleshooting #### Reproduction Create a prefix_route prefix of more than 12 characters and the route will not be chosen. #### Debugging Data There is none. #### Log Messages prefix_route does not seem to produce DEBUG level messages. ### Additional Information ``` kamailio 5.8.6 (x86_64/linux) ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4218 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4218(a)github.com>

3 4

[kamailio/kamailio] topoh: fix bug in hiding record routes in reply messages (PR #4170)
by Farzaneh Soltanzadeh 21 Jun '25

21 Jun '25

#### Pre-Submission Checklist    - [ ] Commit message has the format required by CONTRIBUTING guide - [ ] Commits are split per component (core, individual modules, libs, utils, ...) - [ ] Each component has a single commit (if not, squash them into one commit) - [ ] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [ ] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [ ] PR should be backported to stable branches - [ ] Tested changes locally - [ ] Related to issue #XXXX (replace XXXX with an open issue number) #### Description  You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/4170 -- Commit Summary -- * hide all record routes except last one in reply messages -- File Changes -- M src/modules/topoh/th_msg.c (16) -- Patch Links -- https://github.com/kamailio/kamailio/pull/4170.patch https://github.com/kamailio/kamailio/pull/4170.diff -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/4170 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/pull/4170(a)github.com>

3 13

[kamailio/kamailio] Kamailio blocks in SSL_accept (Issue #4185)
by Mathias Schneuwly 20 Jun '25

20 Jun '25

schneuwlym created an issue (kamailio/kamailio#4185) ### Description This issue is a follow up of the discussion in the [mailing list](https://lists.kamailio.org/mailman3/hyperkitty/list/sr-users@lists.ka… We are using Kamailio 5.8.4 in a yocto scarthgap based image. We noticed on multiple nodes, that suddenly phones lost their registrations and the log showed messages like `send_fd(): sendmsg failed sending 55 on 52: Too many references: cannot splice (109)`. After debugging we guess that the `send_fd` messages are just a symptom, not the cause. Generating a backtrace using `kamctl trap` shows that all TCP receiver are waiting for a `pthread_rwlock_...` lock. Our deployments, where we randomly see the issue, are very small; 10-30 registrations. I've tested kamailio version 5.8.4, 5.8.5, 5.8.6 and 6.0.1 and I was able to reproduce the issue with all versions. I've also tested version 5.5.4, but with an older yocto (dunfell) which used openssl 1.1.1w, **which did not block!** Yocto Scarthgap uses OpenSSL 3.2.3 and we also updated to version 3.4.1, which didn't show any difference. ### Troubleshooting #### Reproduction Meanwhile we found a way to reproduce the issue within seconds. It requires a bit of brute force, but it locks kamailio reliable. We are generating 3 parallel SIP REGISTER streams with sipexer to 3 different accounts. Following an example of one stream: ``` watch -n 0.1 "./sipexer -register -vl 3 -co -com -ex 3600 -fu "2" -au "2" -ap "test" -tk <path to key>.key -tc <path to certificate>.cer -ti -sd -ru sip:<domain> -cu sip:2@<domain> tls:<fqdn>:5061;sleep 0.1;./sipexer -register -vl 3 -co -com -ex 0 -fu "2" -au "2" -ap "test" -tk <path to key>.key -tc <path to cert>.cer -ti -sd -ru sip:<domain> -cu sip:2@<domain> tls:<fqdn>:5061" ``` Our dialplan is quite complex, but I was also able to reproduce the issue with the default [dialplan](https://raw.githubusercontent.com/kamailio/kamailio/refs/heads/5.… with the following changes: ``` 169a170,171 > #!define WITH_TLS > 205a208,219 > alias=udp:<domain>:5060 > alias=udp:127.0.0.1:5060 > alias=udp:10.0.6.1:5060 > alias=tls:<domain> > alias=tls:10.0.6.1:5061 > alias=tls:10.165.42.76:5061 > alias=tls:<domain>:5061 > alias=udp:<domain> > listen=udp:127.0.0.1:5060 > listen=tls:10.0.6.1:5061 > listen=udp:10.0.6.1:5060 > listen=tls:10.165.42.76:5061 498c512 < modparam("tls", "config", "/usr/local/etc/kamailio/tls.cfg") --- > modparam("tls", "config", "/etc/kamailio/kamailio-tls.cfg") ``` ``` # cat /etc/kamailio/kamailio-tls.cfg [server:default] method = TLSv1+ verify_certificate = yes require_certificate = yes private_key = "<path to key>.key" certificate = "<path to cert>.crt" ca_list = "<path to ca>.pem" [client:default] method = TLSv1+ verify_certificate = yes require_certificate = yes private_key = "<path to key>.key" certificate = "<path to cert>.crt" ca_list = "<path to ca>.pem" ``` #### Debugging Data Following the backtrace of one tcp listener process. The full backtrace can be found [here](https://github.com/user-attachments/files/19430313/gdb_kamailio_20250… ``` ---start 4608 ----------------------------------------------------- [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". __futex_abstimed_wait_common (futex_word=futex_word@entry=0x7fc855758a0c, expected=expected@entry=3, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>, cancel=cancel@entry=false) at futex-internal.c:103 warning: 103 futex-internal.c: No such file or directory #0 __futex_abstimed_wait_common (futex_word=futex_word@entry=0x7fc855758a0c, expected=expected@entry=3, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>, cancel=cancel@entry=false) at futex-internal.c:103 err = -512 clockbit = 256 op = <optimized out> #1 0x00007fc85a59429c in __GI___futex_abstimed_wait64 (futex_word=futex_word@entry=0x7fc855758a0c, expected=expected@entry=3, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>) at futex-internal.c:128 No locals. #2 0x00007fc85a59d830 in __pthread_rwlock_wrlock_full64 (abstime=0x0, clockid=0, rwlock=0x7fc855758a00) at /usr/src/debug/glibc/2.39+git/nptl/pthread_rwlock_common.c:730 private = <optimized out> wf = <optimized out> err = <optimized out> prefer_writer = false may_share_futex_used_flag = true wpf = <optimized out> ready = <optimized out> r = <optimized out> may_share_futex_used_flag = <optimized out> r = <optimized out> done = <optimized out> wpf = <optimized out> ready = <optimized out> __value = <optimized out> prefer_writer = <optimized out> private = <optimized out> wf = <optimized out> err = <optimized out> w = <optimized out> w = <optimized out> private = <optimized out> err = <optimized out> w = <optimized out> wf = <optimized out> wf = <optimized out> __value = <optimized out> #3 ___pthread_rwlock_wrlock (rwlock=0x7fc855758a00) at pthread_rwlock_wrlock.c:26 result = <optimized out> #4 0x00007fc859820b49 in CRYPTO_THREAD_write_lock (lock=<optimized out>) at /usr/src/debug/openssl/3.2.3/crypto/threads_pthread.c:108 No locals. #5 0x00007fc8599bd5da in ossl_prov_drbg_generate (drbg=0x7fc855710040, out=0x7fc856471980 "", outlen=48, strength=256, prediction_resistance=0, adin=0x7ffeec8b67e0 "@", adinlen=8) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:645 fork_id = <optimized out> reseed_required = 0 ret = 0 __func__ = "ossl_prov_drbg_generate" err = <optimized out> now = <optimized out> #6 0x00007fc8599bd9ca in ossl_drbg_get_seed (vdrbg=<optimized out>, pout=0x7ffeec8b6880, entropy=<optimized out>, min_len=<optimized out>, max_len=<optimized out>, prediction_resistance=0, adin=0x7ffeec8b6838 "\220'\037V\310\177", adin_len=8) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:175 drbg = 0x7fc855710040 bytes_needed = 48 buffer = 0x7fc856471980 "" __func__ = "ossl_drbg_get_seed" #7 0x00007fc8599bc907 in get_entropy (drbg=<optimized out>, drbg@entry=0x7fc8561f2790, pout=pout@entry=0x7ffeec8b6880, entropy=<optimized out>, min_len=48, max_len=48, prediction_resistance=prediction_resistance@entry=0) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:245 bytes = <optimized out> p_str = 256 __func__ = "get_entropy" #8 0x00007fc8599bd26c in ossl_prov_drbg_reseed_unlocked (drbg=drbg@entry=0x7fc8561f2790, prediction_resistance=prediction_resistance@entry=0, ent=ent@entry=0x0, ent_len=ent_len@entry=0, adin=adin@entry=0x0, adinlen=adinlen@entry=0) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:573 entropy = 0x0 entropylen = 0 __func__ = "ossl_prov_drbg_reseed_unlocked" #9 0x00007fc8599bd68e in ossl_prov_drbg_generate (adinlen=0, adin=0x0, prediction_resistance=0, strength=<optimized out>, outlen=32, out=0x7fc8564717c8 "", drbg=0x7fc8561f2790) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:697 fork_id = <optimized out> reseed_required = <optimized out> ret = <optimized out> fork_id = <optimized out> reseed_required = <optimized out> ret = <optimized out> __func__ = "ossl_prov_drbg_generate" err = <optimized out> now = <optimized out> #10 ossl_prov_drbg_generate (drbg=0x7fc8561f2790, out=0x7fc8564717c8 "", outlen=32, strength=<optimized out>, prediction_resistance=0, adin=0x0, adinlen=0) at /usr/src/debug/openssl/3.2.3/providers/implementations/rands/drbg.c:634 fork_id = <optimized out> reseed_required = 0 ret = 0 __func__ = "ossl_prov_drbg_generate" err = <optimized out> now = <optimized out> #11 0x00007fc8597e75c6 in evp_rand_generate_locked (ctx=ctx@entry=0x7fc8561f26f0, out=out@entry=0x7fc8564717c8 "", outlen=outlen@entry=32, strength=strength@entry=0, prediction_resistance=<optimized out>, prediction_resistance@entry=0, addin=0x0, addin_len=0) at /usr/src/debug/openssl/3.2.3/crypto/evp/evp_rand.c:570 chunk = 32 max_request = 65536 params = {{key = 0x7fc8599e9521 "max_request", data_type = 2, data = 0x7ffeec8b6968, data_size = 8, return_size = 8}, {key = 0x0, data_type = 0, data = 0x0, data_size = 0, return_size = 0}} __func__ = "evp_rand_generate_locked" #12 0x00007fc8597e840b in EVP_RAND_generate (ctx=0x7fc8561f26f0, out=out@entry=0x7fc8564717c8 "", outlen=outlen@entry=32, strength=strength@entry=0, prediction_resistance=prediction_resistance@entry=0, addin=addin@entry=0x0, addin_len=0) at /usr/src/debug/openssl/3.2.3/crypto/evp/evp_rand.c:592 res = <optimized out> #13 0x00007fc8598f9282 in RAND_bytes_ex (ctx=warning: could not convert 'ossl_lib_ctx_st' from the host encoding (ANSI_X3.4-1968) to UTF-32. This normally should not happen, please file a bug report. 0x0, buf=0x7fc8564717c8 "", num=32, strength=0) at /usr/src/debug/openssl/3.2.3/crypto/rand/rand_lib.c:380 rand = <optimized out> meth = <optimized out> __func__ = "RAND_bytes_ex" #14 0x00007fc859bdb62c in def_generate_session_id (ssl=ssl@entry=0x7fc856436e50, id=id@entry=0x7fc8564717c8 "", id_len=id_len@entry=0x7ffeec8b6ae4) at /usr/src/debug/openssl/3.2.3/ssl/ssl_sess.c:326 retry = 0 #15 0x00007fc859bdc132 in ssl_generate_session_id (s=s@entry=0x7fc856436e50, ss=ss@entry=0x7fc856471570) at /usr/src/debug/openssl/3.2.3/ssl/ssl_sess.c:409 tmp = 32 cb = 0x7fc859bdb600 <def_generate_session_id> ssl = <optimized out> __func__ = "ssl_generate_session_id" #16 0x00007fc859bdc31e in ssl_get_new_session (s=s@entry=0x7fc856436e50, session=session@entry=1) at /usr/src/debug/openssl/3.2.3/ssl/ssl_sess.c:464 ss = 0x7fc856471570 __func__ = "ssl_get_new_session" #17 0x00007fc859c3f169 in tls_early_post_process_client_hello (s=0x7fc856436e50) at /usr/src/debug/openssl/3.2.3/ssl/statem/statem_srvr.c:1898 i = <optimized out> loop = <optimized out> sctx = 0x7fc855d24280 j = <optimized out> dgrd = DOWNGRADE_NONE protverr = <optimized out> c = <optimized out> clienthello = 0x7fc856470a90 al = 80 id = <optimized out> comp = 0x0 ciphers = 0x7fc856471330 scsvs = 0x7fc8564713c0 ssl = 0x7fc856436e50 j = <optimized out> i = <optimized out> al = <optimized out> protverr = <optimized out> loop = <optimized out> id = <optimized out> comp = <optimized out> c = <optimized out> ciphers = <optimized out> scsvs = <optimized out> clienthello = <optimized out> dgrd = <optimized out> sctx = <optimized out> ssl = <optimized out> __func__ = "tls_early_post_process_client_hello" err = <optimized out> cipher = <optimized out> trc_out = <optimized out> pos = <optimized out> pref_cipher = <optimized out> master_key_length = <optimized out> m = <optimized out> comp_id = <optimized out> k = <optimized out> m = <optimized out> nn = <optimized out> v = <optimized out> done = <optimized out> o = <optimized out> #18 tls_post_process_client_hello (s=0x7fc856436e50, wst=<optimized out>) at /usr/src/debug/openssl/3.2.3/ssl/statem/statem_srvr.c:2295 rv = <optimized out> cipher = <optimized out> ssl = 0x7fc856436e50 err = <optimized out> __func__ = "tls_post_process_client_hello" #19 0x00007fc859c2bc1b in read_state_machine (s=0x7fc856436e50) at /usr/src/debug/openssl/3.2.3/ssl/statem/statem.c:712 mt = 1 pkt = {curr = 0x7fc856466125 "", remaining = 0} max_message_size = 0x7fc859c3e3e0 <ossl_statem_server_max_message_size> ret = <optimized out> process_message = 0x7fc859c42400 <ossl_statem_server_process_message> post_process_message = 0x7fc859c414f0 <ossl_statem_server_post_process_message> st = 0x7fc856436ee0 len = 273 transition = 0x7fc859c3d330 <ossl_statem_server_read_transition> cb = 0x7fc859c91d90 <sr_ssl_ctx_info_callback> ssl = 0x7fc856436e50 st = <optimized out> ret = <optimized out> mt = <optimized out> len = <optimized out> transition = <optimized out> pkt = <optimized out> process_message = <optimized out> post_process_message = <optimized out> max_message_size = <optimized out> cb = <optimized out> ssl = <optimized out> __func__ = "read_state_machine" #20 state_machine (s=0x7fc856436e50, server=1) at /usr/src/debug/openssl/3.2.3/ssl/statem/statem.c:478 buf = 0x0 cb = 0x7fc859c91d90 <sr_ssl_ctx_info_callback> st = 0x7fc856436ee0 ret = -1 ssret = <optimized out> ssl = 0x7fc856436e50 __func__ = "state_machine" #21 0x00007fc859cbf036 in tls_accept (c=c@entry=0x7fc856461be0, error=error@entry=0x7ffeec8b6dd8) at tls_server.c:457 ret = <optimized out> ssl = 0x7fc856436e50 cert = <optimized out> tls_c = 0x7fc8564171b0 tls_log = <optimized out> __func__ = "tls_accept" ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.4 (x86_64/linux) f1d83d flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: f1d83d compiled on 10:25:58 Nov 12 2024 with x86_64-ruag-linux-gcc 13.3.0 ``` * **Operating System**: It is a yocto scarthgap based image (Version 5.0.6 LTS) ``` Linux ttel 6.6.54-yocto-standard #1 SMP PREEMPT_DYNAMIC Tue Oct 8 03:03:40 UTC 2024 x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4185 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4185(a)github.com>

6 11

[kamailio/kamailio] ims_diameter_server not freeing shared memory (Issue #4116)
by MGrimby 19 Jun '25

19 Jun '25

4 6

[kamailio/kamailio] Memory usage increases everytime tls.reload is executed (Issue #3823)
by Sebastian Denz 18 Jun '25

18 Jun '25

### Description We are using Kamailio 5.7.4 on Debian 12 (from http://deb.kamailio.org/kamailio57) with rtpengine as an Edgeproxy for our clients. The instance terminates SIP/TLS (with Cliencertificates) and forwards the SIP Traffic to internal systems. After some days we are getting errors like this `tls_complete_init(): tls: ssl bug #1491 workaround: not enough memory for safe operation: shm=7318616 threshold1=8912896` First we thought Kamailio just doesnt have enough memory, so we doubled it.. But after some days the Logmessage (and Userissues) occured again. So we monitored the shmmem statistics and found that used and max_used are constantly growing til it reaches the limit. As i mentioned we are using client-certificates and so we are also using the CRL feature. We do have a systemd-timer which fetches the CRL every hour and runs 'kamcmd tls.reload' when finished. Our tls.cfg looks like this: ``` [server:default] method = TLSv1.2+ private_key = /etc/letsencrypt/live/hostname.de/privkey.pem certificate = /etc/letsencrypt/live/hostname.de/fullchain.pem ca_list = /etc/kamailio/ca_list.pem ca_path = /etc/kamailio/ca_list.pem crl = /etc/kamailio/combined.crl.pem verify_certificate = yes require_certificate = yes [client:default] verify_certificate = yes require_certificate = yes ``` After testing a bit we found that every time tls.reload is executed Kamailio consumes a bit more memory which eventually leads to all the memory being consumed which leads to issues for our users. See following example: ``` [0][root@edgar-dev:~]# while true ; do /usr/sbin/kamcmd tls.reload ; /usr/sbin/kamcmd core.shmmem ; sleep 1 ; done Ok. TLS configuration reloaded. { total: 268435456 free: 223001520 used: 41352552 real_used: 45433936 max_used: 45445968 fragments: 73 } Ok. TLS configuration reloaded. { total: 268435456 free: 222377960 used: 41975592 real_used: 46057496 max_used: 46069232 fragments: 78 } Ok. TLS configuration reloaded. { total: 268435456 free: 221748664 used: 42604992 real_used: 46686792 max_used: 46698080 fragments: 77 } Ok. TLS configuration reloaded. { total: 268435456 free: 221110832 used: 43242408 real_used: 47324624 max_used: 47335608 fragments: 81 } ^C [130][root@edgar-dev:~]# ``` ### Troubleshooting #### Reproduction Everytime tls.reload is called the memory consumptions grows.. #### Debugging Data  ``` If you let me know what would be interesting for tracking this down, i am happy to provide logs/debugging data! ``` #### Log Messages  ``` If you let me know what would be interesting for tracking this down, i am happy to provide logs/debugging data! ``` #### SIP Traffic SIP doesnt seem to be relevant here ### Possible Solutions Calling tls.reload less often or restart kamailio before memory is consumed ;) ### Additional Information ``` version: kamailio 5.7.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 12.2.0 ``` * **Operating System**: ``` * Debian GNU/Linux 12 (bookworm) * Linux edgar-dev 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3823 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3823(a)github.com>

9 33

[kamailio/kamailio] cmake format options and naming styles (Issue #4075)
by Daniel-Constantin Mierla 18 Jun '25

18 Jun '25

Discussion to define cmake format options and coding/naming styles. It is better to define most of them now to have coherency in the future. Started from the discussion on #4066. Comment with what you would like to have, describing when useful why your proposal is a good/better option. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4075 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4075(a)github.com>

5 12

[kamailio/kamailio] Segfault when using kamailio 5.8.4 as a presence proxy (Issue #4102)
by Joe Mordica 17 Jun '25

17 Jun '25

### Description We are now using kamailio on the edge as a presence proxy to scale presence. We have 3 internal kamailio instances handling the actual subscriptions and publishing. This works well, however, once per 24-36hours we experience a crash on the border proxy instance. We did not experience this when using the border proxy as a presence server (there was no proxying SUBSCRIBE). ### Troubleshooting #### Reproduction I believe this is happening when a customers network randomly experiences issues where it doesn't receive our replies to subscribes (and continues to send a subscribe). Logs are below. #### Debugging Data  Here's the output while replacing any sensitive info in the payload. ``` GNU gdb (Debian 13.1-3) 13.1 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/sbin/kamailio... Reading symbols from /usr/lib/debug/.build-id/ce/c19d0ed1a928e4e25a84ae20d77e9c904a4892.debug... warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing [New LWP 26] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `kamailio -DD -E -m 8000 -M 512 -f /etc/kamailio/kamailio.cfg -P /var/run/kamail'. Program terminated with signal SIGSEGV, Segmentation fault. #0 via_builder (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0, extra_params=0x0, hp=hp@entry=0x7ffed7e3ebb0) at core/msg_translator.c:2910 2910 core/msg_translator.c: No such file or directory. (gdb) bt full #0 via_builder (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0, extra_params=0x0, hp=hp@entry=0x7ffed7e3ebb0) at core/msg_translator.c:2910 via_len = <optimized out> extra_len = <optimized out> line_buf = <optimized out> max_len = <optimized out> via_prefix_len = <optimized out> address_str = <optimized out> port_str = 0x0 send_sock = 0x0 comp_len = <optimized out> comp_name_len = <optimized out> port = <optimized out> proto = <optimized out> ip = {af = 536870912, len = 0, u = {addrl = {534820896, 1586192}, addr32 = {534820896, 0, 1586192, 0}, addr16 = {47136, 8160, 0, 0, 13328, 24, 0, 0}, addr = " \270\340\037\000\000\000\000\0204\030\000\000\000\000"}} from = 0x0 local_addr = {s = {sa_family = 16, sa_data = "#\000\000\000\000\000\020\000\000\000\000\000\000"}, sin = {sin_family = 16, sin_port = 35, sin_addr = {s_addr = 0}, sin_zero = "\020\000\000\000\000\000\000"}, sin6 = {sin6_family = 16, sin6_port = 35, sin6_flowinfo = 0, sin6_addr = {__in6_u = { __u6_addr8 = "\020\000\000\000\000\000\000\000\003\000\000\000\000\000\000", __u6_addr16 = {16, 0, 0, 0, 3, 0, 0, 0}, __u6_addr32 = {16, 0, 3, 0}}}, sin6_scope_id = 3000082688}, sas = {ss_family = 16, __ss_padding = "#\000\000\000\000\000\020\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\000\241\321\262cg\357`@\000\000\000\000\000\000\000X\f-\177!V\000\000\020\016\217R\371\177\000\000\001\000\000\000!V\000\000*\000\000\000\000\000\000\000\001\000\000\000\376\177\000\000\004\360;\177!V\000\000\250,A\177!V\000\000\000\355\343\327\376\177\000\000\000\241\321\262cg\357`h\256\006R\371\177\000", __ss_align = 94701866020241}} con = 0x0 rxavp = 0x0 xname = {s = 0x0, len = -2141140176} __func__ = "via_builder" #1 0x000056217f155030 in create_via_hf (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0) at core/msg_translator.c:3232 via = <optimized out> extra_params = {s = 0x0, len = 0} hp = {host = 0x56217f4fb380 <default_global_address>, port = 0x56217f4fb370 <default_global_port>} sbuf = "\331\023\000\000\001", '\000' <repeats 18 times> --Type <RET> for more, q to quit, c to continue without paging--info locals slen = <optimized out> xparams = <optimized out> id_buf = <optimized out> id_len = 0 __func__ = "create_via_hf" __llevel = <optimized out> __kld = <optimized out> #2 0x000056217f1592f7 in build_req_buf_from_sip_req (msg=msg@entry=0x7ff9529548b0, returned_len=returned_len@entry=0x7ffed7e3eeac, send_info=send_info@entry=0x7ff764d20bb0, mode=mode@entry=128) at core/msg_translator.c:2096 len = 581 new_len = <optimized out> received_len = 0 rport_len = 0 uri_len = 0 via_len = 0 body_delta = 0 line_buf = 0x0 received_buf = 0x0 rport_buf = 0x0 new_buf = 0x0 buf = 0x56217f5866c0 <buf> "SUBSCRIBE sip:74@sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCa"... path_buf = {s = 0x0, len = 0} offset = 0 s_offset = 0 size = <optimized out> via_anchor = 0x7ff952951300 via_lump = <optimized out> via_rm = <optimized out> via_insert_param = 0x0 path_anchor = <optimized out> path_lump = <optimized out> branch = {s = 0x7ff952954ed8 "z9hG4bKc7ed.5819b4295fe83f981a2fc19b7f739d7e.0", len = 46} flags = 262273 --Type <RET> for more, q to quit, c to continue without paging--list udp_mtu = <optimized out> di = {send_sock = 0x0, to = {s = {sa_family = 18608, sa_data = "\225R\371\177\000\000\002\000\023\331\n4\001\r"}, sin = {sin_family = 18608, sin_port = 21141, sin_addr = { s_addr = 32761}, sin_zero = "\002\000\023\331\n4\001\r"}, sin6 = {sin6_family = 18608, sin6_port = 21141, sin6_flowinfo = 32761, sin6_addr = {__in6_u = { __u6_addr8 = "\002\000\023\331\n4\001\r\000\000\000\000\000\000\000", __u6_addr16 = {2, 55571, 13322, 3329, 0, 0, 0, 0}, __u6_addr32 = {3641901058, 218182666, 0, 0}}}, sin6_scope_id = 0}, sas = {ss_family = 18608, __ss_padding = "\225R\371\177\000\000\002\000\023\331\n4\001\r", '\000' <repeats 103 times>, __ss_align = 0}}, id = 0, send_flags = {f = 0, blst_imask = 0}, proto = 0 '\000', proto_pad0 = 0 '\000', proto_pad1 = 0} ret = <optimized out> __func__ = "build_req_buf_from_sip_req" error00 = <optimized out> #3 0x00007ff951fbe90d in prepare_new_uac (t=t@entry=0x7ff764d20880, i_req=i_req@entry=0x7ff9529548b0, branch=branch@entry=0, uri=<optimized out>, uri@entry=0x7ff9529548e8, path=<optimized out>, path@entry=0x7ff952954f60, next_hop=<optimized out>, fsocket=0x7ff9528f0e10, snd_flags=..., fproto=<optimized out>, flags=<optimized out>, instance=<optimized out>, ruid=<optimized out>, location_ua=<optimized out>) at ./src/modules/tm/t_fwd.c:482 shbuf = 0x0 add_rm_backup = <optimized out> body_lumps_backup = <optimized out> parsed_uri_bak = {user = { s = 0x56217f5866ce <buf+14> "74(a)sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCall-ID: 0_18056"..., len = 2}, passwd = {s = 0x0, len = 0}, host = { s = 0x56217f5866d1 <buf+17> "sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCall-ID: 0_18056078"..., len = 11}, port = {s = 0x0, len = 0}, params = {s = 0x0, len = 0}, sip_params = {s = 0x0, len = 0}, headers = { s = 0x0, len = 0}, port_no = 0, proto = 0, type = SIP_URI_T, flags = 0, transport = {s = 0x0, len = 0}, ttl = {s = 0x0, len = 0}, user_param = {s = 0x0, len = 0}, maddr = { s = 0x0, len = 0}, method = {s = 0x0, len = 0}, lr = {s = 0x0, len = 0}, r2 = {s = 0x0, len = 0}, gr = {s = 0x0, len = 0}, transport_val = {s = 0x0, len = 0}, ttl_val = { s = 0x0, len = 0}, user_param_val = {s = 0x0, len = 0}, maddr_val = {s = 0x0, len = 0}, method_val = {s = 0x0, len = 0}, lr_val = {s = 0x0, len = 0}, r2_val = {s = 0x0, len = 0}, gr_val = {s = 0x0, len = 0}} ret = -1 len = 32761 parsed_uri_ok_bak = <optimized out> free_new_uri = 1 msg_uri_bak = {s = 0x0, len = 0} dst_uri_bak = {s = 0x7ff9529525f0 "sip:10.52.1.13:5081", len = 19} dst_uri_backed_up = 1 path_bak = {s = 0x0, len = 0} free_path = 1 instance_bak = {s = 0x0, len = 0} ``` #### Log Messages  ``` ERROR 2025-01-10T06:00:31.847348563Z [resource.labels.containerName: kamailio] 8(28) CRITICAL: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [timer.h:188]: _set_fr_retr(): already added: 0x7ff763e62250 , tl=0x7ff763e62270!!! ERROR 2025-01-10T06:00:31.847388483Z [resource.labels.containerName: kamailio] 8(28) CRITICAL: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [t_fwd.c:1613]: t_send_branch(): BUG: retransmission already started for: 0x7ff763e62250 ERROR 2025-01-10T06:00:31.847394809Z [resource.labels.containerName: kamailio] 8(28) ERROR: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} sl [sl_funcs.c:428]: sl_reply_error(): stateless error reply used: No error (0/SL) ERROR 2025-01-10T06:00:31.847400738Z [resource.labels.containerName: kamailio] 8(28) BUG: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [t_lookup.c:2052]: t_unref(): REQ_ERR DELAYED should have been caught much earlier for 0x7ff763e61f70: 24 (hex 18) ERROR 2025-01-10T06:00:47.346744773Z [resource.labels.containerName: kamailio] 45(65) ERROR: <core> [core/tcp_main.c:4818]: tcpconn_main_timeout(): connect 64.63.142.90:12190 failed (timeout) ERROR 2025-01-10T06:00:58.274915089Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 20 ERROR 2025-01-10T06:01:02.715789076Z [resource.labels.containerName: kamailio] 0(1) ALERT: <core> [main.c:805]: handle_sigs(): child process 26 exited by a signal 11 ERROR 2025-01-10T06:01:02.715827333Z [resource.labels.containerName: kamailio] 0(1) ALERT: <core> [main.c:809]: handle_sigs(): core was generated ERROR 2025-01-10T06:01:02.718815287Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 25 ERROR 2025-01-10T06:01:02.720985493Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 23 ERROR 2025-01-10T06:01:02.725450552Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 30 ``` #### SIP Traffic  ``` (paste your sip traffic here) ``` ### Possible Solutions I can atleast say that when this segfault happens that these logs are always present right before the crash. However, I also see instances of these logs without a crash immediately following. On the proxy we are doing this is withindlg route to send subscribes to the presence route: ``` if KSR.is_SUBSCRIBE() and KSR.is_myself_ruri() then -- in-dialog subscribe requests ksr_route_presence(); KSR.x.exit(); end ``` Any then this is our ksr_route_presence(): ``` function ksr_route_presence() if not KSR.is_SUBSCRIBE() then return 1; end local fuser = KSR.kx.get_fuser(); if fuser=='' then KSR.xlog.xinfo('Subscription attempt without username from IP ' .. KSR.kx.get_srcip() .. ' - Rejecting\n'); KSR.sl.sl_send_reply(404, 'Not Found'); KSR.x.exit(); end KSR.pv.sets('$avp(s:tenant)', string.match(fuser, '-(.*)')); local setId = '6' .. KSR.kx.get_def('DEFAULT_DISPATCH_SET'); local modeId = '7'; if KSR.dispatcher.ds_select_dst(setId, modeId)<0 then KSR.sl.send_reply(404, 'No destination'); KSR.x.exit(); end KSR.corex.set_send_socket(KSR.kx.get_def('PRIVATE_LISTEN_IP') .. ':' .. KSR.kx.get_def('INTERNAL_PORT')); KSR.tm.t_on_failure('ksr_route_rtf_dispatch'); ksr_route_relay(); end ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 12.2.0 ``` * **Operating System**:  ``` PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" ``` ``` Linux gke-us-south1-external-sip-800ca69e-qyb3 5.15.0-1048-gke #53-Ubuntu SMP Tue Nov 28 00:39:01 UTC 2023 x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4102 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4102(a)github.com>

5 34

[kamailio/kamailio] TOPOS removes via header when calling drop in event_route[topos:msg-outgoing] (Issue #4157)
by Brandon Armstead 16 Jun '25

16 Jun '25

CrazyTux created an issue (kamailio/kamailio#4157) ### TOPOS removes via header ``` event_route[topos:msg-outgoing] { # Feature Flag is enabled ** problem occurs here ** if($sht(feature_flags=>kamailio-disable-internal-topos) == "true") { if(ds_is_from_list(KAM_SET_ID, 3, "sip:$sndto(ip):5060")) { drop; } } } ``` ### Troubleshooting - verified via exists in original 200 OK REPLY from Freeswitch - checked wire and verified via header is missing when forwarding 200 OK onward to next kamailio (B) node from Kamailio (A) #### Reproduction Call drop in event route `topos:msg-outgoing` - call flow: Freeswitch -> Kamailio A -> Kamailio B -> UAC - place incoming call into kamailio - put call on hold / reinvite #### Debugging Data ``` Disabling topos entirely via header is correctly sent from A -> B. ``` #### Log Messages (Kamailio B) ``` 2025-02-25T00:19:08.545447+00:00 /usr/sbin/kamailio[3959021]: ERROR: {2 1 INVITE 995df181-4eae-40ae-8225-427ce8252250} <core> [core/receive.c:542]: receive_msg(): no via found in reply 2025-02-25T00:19:09.045917+00:00 /usr/sbin/kamailio[3959037]: ERROR: {2 1 INVITE 995df181-4eae-40ae-8225-427ce8252250} <core> [core/receive.c:542]: receive_msg(): no via found in reply 2025-02-25T00:19:10.046936+00:00 /usr/sbin/kamailio[3958995]: ERROR: {2 1 INVITE 995df181-4eae-40ae-8225-427ce8252250} <core> [core/receive.c:542]: receive_msg(): no via found in reply 2025-02-25T00:19:12.048099+00:00 /usr/sbin/kamailio[3959003]: ERROR: {2 1 INVITE 995df181-4eae-40ae-8225-427ce8252250} <core> [core/receive.c:542]: receive_msg(): no via found in reply ``` #### SIP Traffic `reinvite B -> A:` ``` INVITE sip:mod_sofia@54.245.XX.XX:5060 SIP/2.0 Record-Route: <sip:35.161.XX.XX:5060;r2=on;lr;ftag=9vX27Um099jZS;did=a2b.7e62> Record-Route: <sip:35.161.XX.XX:7443;transport=ws;r2=on;lr;ftag=9vX27Um099jZS;did=a2b.7e62> Via: SIP/2.0/UDP 35.161.XX.XX:5060;branch=z9hG4bK16be.085d2f3d83e8d939f7f1d3fbc4a5ff09.0 Via: SIP/2.0/WSS 74jssm3ooc66.invalid;received=209.160.251.114;branch=z9hG4bK3968143 To: <sip:9496294444@54.245.XX.XX>;tag=9vX27Um099jZS From: <sip:Dff742e5340fDd93847e@9e3f4e45-01cf-1f66-8d70-8d14ec555153>;tag=q1vfbk6o6q CSeq: 1 INVITE Call-ID: 995df181-4eae-40ae-8225-427ce8252250 Max-Forwards: 69 Allow: ACK,BYE,CANCEL,INFO,INVITE,MESSAGE,NOTIFY,OPTIONS,PRACK,REFER,REGISTER,SUBSCRIBE Contact: <sip:tu0r7r7u@74jssm3ooc66.invalid;transport=ws> Supported: outbound User-Agent: WebPhone/3.1740423483.87 Content-Type: application/sdp Content-Length: 1538 Route: <sip:34.213.XX.XX:5060;r2=on;lr;ftag=9vX27Um099jZS;did=a2b.1142>,<sip:34.213.XX.XX;r2=on;lr;ftag=9vX27Um099jZS;did=a2b.1142> P-SR-XUID: atpsh-7cde-67bcbccd-3c68ef-1 v=0 o=- 4757376688759011024 3 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 a=extmap-allow-mixed a=msid-semantic: WMS 35a33e17-dd80-46fa-afe9-72fdd03d4e41 m=audio 32999 UDP/TLS/RTP/SAVPF 0 102 101 103 63 9 8 13 c=IN IP4 209.160.251.114 a=rtcp:9 IN IP4 0.0.0.0 a=candidate:2845667355 1 udp 2122260223 10.82.226.218 54425 typ host generation 0 network-id 1 network-cost 10 a=candidate:1750280017 1 udp 1686052607 209.160.251.114 32999 typ srflx raddr 10.82.226.218 rport 54425 generation 0 network-id 1 network-cost 10 a=ice-ufrag:Eb9h a=ice-pwd:nQdfpsILiUI+YKSQT3eZy52x a=ice-options:trickle a=fingerprint:sha-256 BA:68:25:D5:66:70:A9:17:38:18:74:0A:B3:C2:3E:F9:3A:A0:7C:5F:18:0F:EB:B7:93:28:86:56:62:23:DF:E5 a=setup:actpass a=mid:0 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=sendonly a=msid:35a33e17-dd80-46fa-afe9-72fdd03d4e41 57910219-943b-4687-9511-b9a9d0a1ea6f a=rtcp-mux a=rtcp-rsize a=rtpmap:0 PCMU/8000 a=rtpmap:102 opus/48000/2 a=fmtp:102 minptime=10;useinbandfec=1 a=rtpmap:101 telephone-event/8000 a=rtpmap:103 telephone-event/48000 a=rtpmap:63 red/48000/2 a=fmtp:63 102/102 a=rtpmap:9 G722/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:13 CN/8000 a=ssrc:2604971680 cname:YEcJO5XrSKZor1jY a=ssrc:2604971680 msid:35a33e17-dd80-46fa-afe9-72fdd03d4e41 57910219-943b-4687-9511-b9a9d0a1ea6f ``` `200 OK Freeswitch -> A` ``` SIP/2.0 200 OK v:SIP/2.0/UDP 34.213.XX.XX:5060;branch=z9hG4bK16be.86ee5e1d4190783bd6a79bd21110d190.0 f:<sip:Dff742e5340fDd93847e@9e3f4e45-01cf-1f66-8d70-8d14ec555153>;tag=q1vfbk6o6q t:<sip:9496294444@54.245.XX.XX>;tag=9vX27Um099jZS i:995df181-4eae-40ae-8225-427ce8252250 CSeq:1 INVITE m:<sip:mod_sofia@54.245.XX.XX:5060> Accept:application/sdp Allow:INVITE,ACK,BYE,CANCEL,OPTIONS,MESSAGE,INFO,UPDATE,REGISTER,REFER,NOTIFY,PUBLISH,SUBSCRIBE k:timer,path,replaces x:90;refresher=uas c:application/sdp Content-Disposition:session l:284 v=0 o=FreeSWITCH 1740424963 1740424965 IN IP4 54.245.XX.XX s=FreeSWITCH c=IN IP4 54.245.XX.XX t=0 0 m=audio 17776 RTP/AVP 0 101 a=rtpmap:0 PCMU/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-15 a=recvonly a=ptime:20 a=rtcp-mux a=rtcp:17776 IN IP4 54.245.XX.XX ``` `200 OK A -> B` ``` SIP/2.0 200 OK f:<sip:Dff742e5340fDd93847e@9e3f4e45-01cf-1f66-8d70-8d14ec555153>;tag=q1vfbk6o6q t:<sip:9496294444@54.245.XX.XX>;tag=9vX27Um099jZS i:995df181-4eae-40ae-8225-427ce8252250 CSeq:1 INVITE m:<sip:mod_sofia@54.245.XX.XX:5060> Accept:application/sdp Allow:INVITE,ACK,BYE,CANCEL,OPTIONS,MESSAGE,INFO,UPDATE,REGISTER,REFER,NOTIFY,PUBLISH,SUBSCRIBE k:timer,path,replaces x:90;refresher=uas c:application/sdp Content-Disposition:session l:315 v=0 o=FreeSWITCH 1740424963 1740424965 IN IP4 34.211.XX.XX s=FreeSWITCH c=IN IP4 34.211.XX.XX t=0 0 m=audio 20374 RTP/AVP 0 101 a=mid:0 a=rtpmap:0 PCMU/8000 a=rtpmap:101 telephone-event/8000 a=fmtp:101 0-15 a=recvonly a=rtcp:20374 a=rtcp-mux a=ptime:20 a=ice-options:trickle a=end-of-candidates ``` ### Possible Solutions TBD ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.2 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 10.2.1 ``` <img width="629" alt="Image" src="https://github.com/user-attachments/assets/fc46f76e-7a6f-431f-a00e-84fbbece…" /> * **Operating System**: ``` No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye ``` I can provide PCAP as well if required. Let me know if additional information is required. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4157 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4157(a)github.com>

4 10

[kamailio/kamailio] tcp_timer_check_connections() logs on CRITICAL and is not caught by event_route[tcp:timeout] (Issue #3995)
by Fritjof Höst 13 Jun '25

13 Jun '25

### Description Hi! We are in the progress of upgrading from Kamailio 5.5 to 5.8. During our testing we have noticed a new error being reported from Kamailio. We don’t see any other errors following it. ```jsx /usr/sbin/kamailio[201]: CRITICAL: <core> [core/tcp_main.c:5544]: tcp_timer_check_connections(): message processing timeout on connection id: 67896 (state: 3) - closing ``` It does seem to be [new code](https://github.com/kamailio/kamailio/blob/master/src/core/tcp_main.c#… in Kamailio reporting this issue. Given that this is a fairly expected thing, cleaning up a connection which receives no traffic within the given time, is there a need for it to be reported on CRITICAL? I’d also expect it to be caught by ``` event_route[tcp:timeout] { xlog("L_INFO","connection $conid timeouts (unanswered keepalives)"); } ``` given that [the description](https://www.kamailio.org/docs/modules/stable/modules/tcpops.ht… of this one is `Called for connection timeouts (unanswered keepalives).`. ### Troubleshooting We don't have any way to reproduce it, we are still investigating it to figure out the cause. It happens around every 2 hours, so we think there might some some scheduled test or job running in our test system causing this. ### Additional Information * **Kamailio Version** - 5.8.3 * **Operating System**: Debian 12 AMI from the AWS Marketplace. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3995 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3995(a)github.com>

6 12

[kamailio/kamailio] tls_wolfssl: crash qm_free / qm_debug_check_frag (Issue #4074)
by Fred Posner 12 Jun '25

12 Jun '25

### Introduction First, I apologize for the lack of detail here. I am unable to get a core dump and as a production box, I was unable to run in debug mode. ### Description Using Kamailio 5.8.4 on Debian 12, every 60-80 minutes, the software would crash, blaming tls_wolfssl, such as: ``` 2024-12-16T14:58:27.554497+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring 2024-12-16T14:58:27.554555+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f3440ba7560 (address 0x7f3440ba75a0) beginning overwritten (23)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by :29. Exec from core/mem/q_malloc.c:546. 2024-12-16T14:58:27.616558+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9089]: CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 103 2024-12-16T14:58:27.617684+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT: <core> [main.c:805]: handle_sigs(): child process 9084 exited by a signal 6 2024-12-16T14:58:27.618339+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT: <core> [main.c:809]: handle_sigs(): core was not generated 2024-12-16T14:58:27.618477+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: INFO: <core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD 2024-12-16T14:58:27.618589+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9088]: INFO: <core> [main.c:888]: sig_usr(): signal 15 received ``` ``` 2024-12-16T14:57:52.029277+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring 2024-12-16T14:57:52.029347+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f085996b500 (address 0x7f085996b540) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546. 2024-12-16T14:57:52.048597+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21611]: CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 101 2024-12-16T14:57:52.048921+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT: <core> [main.c:805]: handle_sigs(): child process 21605 exited by a signal 6 2024-12-16T14:57:52.048953+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT: <core> [main.c:809]: handle_sigs(): core was not generated 2024-12-16T14:57:52.048975+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: INFO: <core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD 2024-12-16T14:57:52.048997+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21609]: INFO: <core> [main.c:888]: sig_usr(): signal 15 received ``` ``` 2024-12-16T16:21:35.967576+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[10591]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f442778d110 (address 0x7f442778d150) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546. ``` ``` 2024-12-16T13:31:55.995222+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7602]: CRITICAL: <core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer (0x7f8e55830a70), called from tls_wolfssl: tls_init.c: ser_free(240), first free tls_wolfssl: tls_init.c: ser_malloc(228) - ignoring 2024-12-16T13:32:40.093954+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL: <core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x7a100000000 (out of memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring 2024-12-16T13:32:40.094047+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546. 2024-12-16T13:32:40.260913+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL: <core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer (0x7f8e55761a70), called from core: core/tcp_main.c: _tcpconn_free(1666), first free tls_wolfssl: tls_init.c: ser_free(240) - ignoring 2024-12-16T13:32:40.263520+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL: <core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from core/mem/q_malloc.c:546. ``` The system was handling ~900 TLS registrations with ~2700 presence SUBSCRIBE events. **Running the same config with `tls.so` does not result in crash.** ### Additional Information * **Kamailio Version** - 5.8.4 ``` version: kamailio 5.8.4 (x86_64/linux) 598105-dirty flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 598105 -dirty compiled on 04:31:34 Dec 16 2024 with gcc 12.2.0 ``` * **Operating System**: ``` Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm Linux ip-172-16-0-52 6.1.0-28-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1 (2024-11-22) x86_64 GNU/Linux ``` * TCP Settings ``` children=16 disable_tcp=no tcp_connection_lifetime=3605 tcp_accept_no_cl=yes tcp_max_connections=8192 tcp_defer_accept=yes tcp_accept_unique=0 tcp_connection_lifetime=3605 tcp_connection_match=1 tcp_connect_timeout=10 tcp_crlf_ping=yes tcp_linger2=1 tcp_keepalive=yes tcp_reuse_port=yes tcp_keepidle=20 tcp_keepintvl=15 tcp_rd_buf_size=65536 tcp_msg_data_timeout=40 tcp_msg_read_timeout=40 tcp_check_timer=20 tls_threads_mode=2 tls_max_connections=8192 ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4074 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4074(a)github.com>

3 6

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

sr-dev April 2025