sr-dev January 2025

sr-dev@lists.kamailio.org

21 participants
295 discussions

[kamailio/kamailio] clean_sa(): Error sending delete SAs command via netlink socket: Invalid argument (#2707)
by openspring 16 Apr '25

16 Apr '25

### Description  #### Log Messages  ``` 0(117561) DEBUG: <core> [core/route_struct.c:128]: mk_action(): ACTION type5, i/count 0/1: type 1(1)/ data 0x7f8583620ef0 0(117561) DEBUG: <core> [core/route_struct.c:128]: mk_action(): ACTION type16, i/count 0/3: type 22(16)/ data 0x7f8583621ad8 0(117561) DEBUG: <core> [core/route_struct.c:128]: mk_action(): ACTION type16, i/count 1/3: type 8(8)/ data 0x7f8583620fd8 0(117561) DEBUG: <core> [core/route_struct.c:128]: mk_action(): ACTION type16, i/count 2/3: type 0(0)/ data 0x7f8500000000 0(117561) DEBUG: <core> [core/route.c:129]: route_add(): mapping routing block (0xb914c0)[MT_indialog_reply] to 6 ERROR: bad config file (1 errors) 0(117561) DEBUG: <core> [core/sr_module.c:765]: destroy_modules(): starting modules destroy phase 0(117561) DEBUG: debugger [debugger_api.c:1267]: dbg_destroy_mod_levels(): Destroyed _dbg_mod_table, size 32 0(117561) ERROR: ims_ipsec_pcscf [ipsec.c:567]: clean_sa(): Error sending delete SAs command via netlink socket: Invalid argument 0(117561) WARNING: ims_ipsec_pcscf [cmd.c:998]: ipsec_cleanall(): Error cleaning IPSec Security associations during startup. 0(117561) ERROR: ims_ipsec_pcscf [ipsec.c:609]: clean_policy(): Error sending delete policies command via netlink socket: Invalid argument 0(117561) WARNING: ims_ipsec_pcscf [cmd.c:1002]: ipsec_cleanall(): Error cleaning IPSec Policies during startup. 0(117561) ERROR: ims_ipsec_pcscf [ims_ipsec_pcscf_mod.c:309]: mod_destroy(): Error destroying spi generator 0(117561) ERROR: ims_ipsec_pcscf [ims_ipsec_pcscf_mod.c:313]: mod_destroy(): Error destroying port generator 0(117561) DEBUG: tm [t_funcs.c:84]: tm_shutdown(): start 0(117561) DEBUG: tm [t_funcs.c:87]: tm_shutdown(): emptying hash table 0(117561) DEBUG: tm [t_funcs.c:89]: tm_shutdown(): removing semaphores 0(117561) DEBUG: tm [t_funcs.c:91]: tm_shutdown(): destroying tmcb lists 0(117561) DEBUG: tm [t_funcs.c:94]: tm_shutdown(): done 0(117561) INFO: <core> [core/sctp_core.c:53]: sctp_core_destroy(): SCTP API not initialized 0(117561) DEBUG: <core> [core/mem/shm.c:289]: shm_destroy_manager(): destroying memory manager: q_malloc 0(117561) DEBUG: <core> [core/mem/q_malloc.c:1185]: qm_shm_lock_destroy(): destroying the shared memory lock 0(117561) DEBUG: <core> [core/mem/pkg.c:98]: pkg_destroy_manager(): destroying memory manager: q_malloc ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.4.4 (x86_64/linux) 8d6c2b flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 8d6c2b compiled on 22:26:30 Apr 11 2021 with gcc 4.8.5 ``` * **Operating System**:  ``` NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2707

5 5

[kamailio/kamailio] tcp_timer_check_connections() logs on CRITICAL and is not caught by event_route[tcp:timeout] (Issue #3995)
by Fritjof Höst 16 Apr '25

16 Apr '25

### Description Hi! We are in the progress of upgrading from Kamailio 5.5 to 5.8. During our testing we have noticed a new error being reported from Kamailio. We don’t see any other errors following it. ```jsx /usr/sbin/kamailio[201]: CRITICAL: <core> [core/tcp_main.c:5544]: tcp_timer_check_connections(): message processing timeout on connection id: 67896 (state: 3) - closing ``` It does seem to be [new code](https://github.com/kamailio/kamailio/blob/master/src/core/tcp_main.c#… in Kamailio reporting this issue. Given that this is a fairly expected thing, cleaning up a connection which receives no traffic within the given time, is there a need for it to be reported on CRITICAL? I’d also expect it to be caught by ``` event_route[tcp:timeout] { xlog("L_INFO","connection $conid timeouts (unanswered keepalives)"); } ``` given that [the description](https://www.kamailio.org/docs/modules/stable/modules/tcpops.ht… of this one is `Called for connection timeouts (unanswered keepalives).`. ### Troubleshooting We don't have any way to reproduce it, we are still investigating it to figure out the cause. It happens around every 2 hours, so we think there might some some scheduled test or job running in our test system causing this. ### Additional Information * **Kamailio Version** - 5.8.3 * **Operating System**: Debian 12 AMI from the AWS Marketplace. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3995 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3995(a)github.com>

5 8

[kamailio/kamailio] fr_timer documentation mistake (Issue #3939)
by Dmitry Sinina 05 Apr '25

05 Apr '25

Documentation https://www.kamailio.org/docs/modules/devel/modules/tm.html#tm.p.fr_timer says: >Timer which hits if no **final reply** for a request or ACK for a negative INVITE reply arrives (in milliseconds). Looks like it is not correct description - fr_timer value works until 1xx(not **final**) response received and then timer restarted with fr_inv_timer value there: https://github.com/kamailio/kamailio/blob/master/src/modules/tm/t_reply.c#L… Looks like old doc from sip router https://sip-router.org/wiki/ref_manual/timers explains it correctly: > fr_timer This timer is used for all SIP requests. It hits if no reply for an INVITE request or other request has been received (F in milliseconds). If a provisional reply is received for an INVITE (any 1xx), then the fr_inv_timer will be used instead. And if no replies (at all) for an INVITE are received before `fr_timer` hits, the transaction is terminated with a 408 in failure route. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3939 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3939(a)github.com>

3 8

[kamailio/kamailio] tls: Create one shared context and reuse it. (PR #3972)
by Xenofon Karamanos 01 Apr '25

01 Apr '25

#### Pre-Submission Checklist    - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [ ] Small bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [ ] PR should be backported to stable branches - [x] Tested changes locally - [x] Related to issue #3823 #### Description  This PR aims to implement what was discussed in [mailing list](https://lists.kamailio.org/mailman3/hyperkitty/list/sr-dev@lists.kama… regarding some `tls.reload` and increasing memory usage. It adds a new parameter `enable_shared_ctx` in `tls` module that if set to 0, preserves the old behavior and if set to 1 (other than 0 tbh), it creates a single SSL context that is being shared. This have the effect of using way less memory when initialized as well, but also minimizes (can't say it fixes the problem) the `tls.reload` memory increase. I have also added a small markdown (comparison.md) file, where some comparisons where made between enabled/disabled shared context and with/without CA file (where the initial problem was occurring by the reporter). Feedback would be necessary to verify whether this patch, acts as expected and kamailio works as intented. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/3972 -- Commit Summary -- * tls: Add parameter for shared contexts * tls: Comparison for enable_shared_ctx -- File Changes -- A comparison.md (15) M src/modules/tls/tls_domain.c (172) M src/modules/tls/tls_mod.c (11) M src/modules/tls/tls_mod.h (1) -- Patch Links -- https://github.com/kamailio/kamailio/pull/3972.patch https://github.com/kamailio/kamailio/pull/3972.diff -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/3972 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/pull/3972(a)github.com>

4 9

[kamailio/kamailio] core: local TCP socket is bound on listening address (PR #3925)
by sergey-safarov 01 Apr '25

01 Apr '25

#### Pre-Submission Checklist - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [x] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist: - [x] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue #XXXX (replace XXXX with an open issue number) #### Description As a local IP address for TCP sending operation the Kamailio service is taking the same network_interface/IP_address, which is used by the service for TCP listening. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/3925 -- Commit Summary -- * core: local TCP socket is bound on listening address -- File Changes -- M src/core/tcp_main.c (24) -- Patch Links -- https://github.com/kamailio/kamailio/pull/3925.patch https://github.com/kamailio/kamailio/pull/3925.diff -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/3925 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/pull/3925(a)github.com>

5 14

[kamailio/kamailio] Allow calling `route()` in `onsend_route` (Issue #4130)
by Ben Kaufman 28 Mar '25

28 Mar '25

### Description Currently calling `route()` within the `onsend_route` will cause kamailio to fail to start with an error that `command not allowed in onsend_route` ### Expected behavior While there is a limited amount of functions that can be allowed within the `onsend_route`, calling `route()` here would be done simply for code organization. #### Actual observed behavior Kamailio fails to start; ### Possible Solutions Allow `route()` to be called from `onsend_route`. -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4130 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4130(a)github.com>

2 2

[kamailio/kamailio] Memory usage increases everytime tls.reload is executed (Issue #3823)
by Sebastian Denz 27 Mar '25

27 Mar '25

### Description We are using Kamailio 5.7.4 on Debian 12 (from http://deb.kamailio.org/kamailio57) with rtpengine as an Edgeproxy for our clients. The instance terminates SIP/TLS (with Cliencertificates) and forwards the SIP Traffic to internal systems. After some days we are getting errors like this `tls_complete_init(): tls: ssl bug #1491 workaround: not enough memory for safe operation: shm=7318616 threshold1=8912896` First we thought Kamailio just doesnt have enough memory, so we doubled it.. But after some days the Logmessage (and Userissues) occured again. So we monitored the shmmem statistics and found that used and max_used are constantly growing til it reaches the limit. As i mentioned we are using client-certificates and so we are also using the CRL feature. We do have a systemd-timer which fetches the CRL every hour and runs 'kamcmd tls.reload' when finished. Our tls.cfg looks like this: ``` [server:default] method = TLSv1.2+ private_key = /etc/letsencrypt/live/hostname.de/privkey.pem certificate = /etc/letsencrypt/live/hostname.de/fullchain.pem ca_list = /etc/kamailio/ca_list.pem ca_path = /etc/kamailio/ca_list.pem crl = /etc/kamailio/combined.crl.pem verify_certificate = yes require_certificate = yes [client:default] verify_certificate = yes require_certificate = yes ``` After testing a bit we found that every time tls.reload is executed Kamailio consumes a bit more memory which eventually leads to all the memory being consumed which leads to issues for our users. See following example: ``` [0][root@edgar-dev:~]# while true ; do /usr/sbin/kamcmd tls.reload ; /usr/sbin/kamcmd core.shmmem ; sleep 1 ; done Ok. TLS configuration reloaded. { total: 268435456 free: 223001520 used: 41352552 real_used: 45433936 max_used: 45445968 fragments: 73 } Ok. TLS configuration reloaded. { total: 268435456 free: 222377960 used: 41975592 real_used: 46057496 max_used: 46069232 fragments: 78 } Ok. TLS configuration reloaded. { total: 268435456 free: 221748664 used: 42604992 real_used: 46686792 max_used: 46698080 fragments: 77 } Ok. TLS configuration reloaded. { total: 268435456 free: 221110832 used: 43242408 real_used: 47324624 max_used: 47335608 fragments: 81 } ^C [130][root@edgar-dev:~]# ``` ### Troubleshooting #### Reproduction Everytime tls.reload is called the memory consumptions grows.. #### Debugging Data  ``` If you let me know what would be interesting for tracking this down, i am happy to provide logs/debugging data! ``` #### Log Messages  ``` If you let me know what would be interesting for tracking this down, i am happy to provide logs/debugging data! ``` #### SIP Traffic SIP doesnt seem to be relevant here ### Possible Solutions Calling tls.reload less often or restart kamailio before memory is consumed ;) ### Additional Information ``` version: kamailio 5.7.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 12.2.0 ``` * **Operating System**: ``` * Debian GNU/Linux 12 (bookworm) * Linux edgar-dev 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/3823 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/3823(a)github.com>

8 30

[kamailio/kamailio] Segfault when using kamailio 5.8.4 as a presence proxy (Issue #4102)
by Joe Mordica 27 Mar '25

27 Mar '25

### Description We are now using kamailio on the edge as a presence proxy to scale presence. We have 3 internal kamailio instances handling the actual subscriptions and publishing. This works well, however, once per 24-36hours we experience a crash on the border proxy instance. We did not experience this when using the border proxy as a presence server (there was no proxying SUBSCRIBE). ### Troubleshooting #### Reproduction I believe this is happening when a customers network randomly experiences issues where it doesn't receive our replies to subscribes (and continues to send a subscribe). Logs are below. #### Debugging Data  Here's the output while replacing any sensitive info in the payload. ``` GNU gdb (Debian 13.1-3) 13.1 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/sbin/kamailio... Reading symbols from /usr/lib/debug/.build-id/ce/c19d0ed1a928e4e25a84ae20d77e9c904a4892.debug... warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing [New LWP 26] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `kamailio -DD -E -m 8000 -M 512 -f /etc/kamailio/kamailio.cfg -P /var/run/kamail'. Program terminated with signal SIGSEGV, Segmentation fault. #0 via_builder (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0, extra_params=0x0, hp=hp@entry=0x7ffed7e3ebb0) at core/msg_translator.c:2910 2910 core/msg_translator.c: No such file or directory. (gdb) bt full #0 via_builder (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0, extra_params=0x0, hp=hp@entry=0x7ffed7e3ebb0) at core/msg_translator.c:2910 via_len = <optimized out> extra_len = <optimized out> line_buf = <optimized out> max_len = <optimized out> via_prefix_len = <optimized out> address_str = <optimized out> port_str = 0x0 send_sock = 0x0 comp_len = <optimized out> comp_name_len = <optimized out> port = <optimized out> proto = <optimized out> ip = {af = 536870912, len = 0, u = {addrl = {534820896, 1586192}, addr32 = {534820896, 0, 1586192, 0}, addr16 = {47136, 8160, 0, 0, 13328, 24, 0, 0}, addr = " \270\340\037\000\000\000\000\0204\030\000\000\000\000"}} from = 0x0 local_addr = {s = {sa_family = 16, sa_data = "#\000\000\000\000\000\020\000\000\000\000\000\000"}, sin = {sin_family = 16, sin_port = 35, sin_addr = {s_addr = 0}, sin_zero = "\020\000\000\000\000\000\000"}, sin6 = {sin6_family = 16, sin6_port = 35, sin6_flowinfo = 0, sin6_addr = {__in6_u = { __u6_addr8 = "\020\000\000\000\000\000\000\000\003\000\000\000\000\000\000", __u6_addr16 = {16, 0, 0, 0, 3, 0, 0, 0}, __u6_addr32 = {16, 0, 3, 0}}}, sin6_scope_id = 3000082688}, sas = {ss_family = 16, __ss_padding = "#\000\000\000\000\000\020\000\000\000\000\000\000\000\003\000\000\000\000\000\000\000\000\241\321\262cg\357`@\000\000\000\000\000\000\000X\f-\177!V\000\000\020\016\217R\371\177\000\000\001\000\000\000!V\000\000*\000\000\000\000\000\000\000\001\000\000\000\376\177\000\000\004\360;\177!V\000\000\250,A\177!V\000\000\000\355\343\327\376\177\000\000\000\241\321\262cg\357`h\256\006R\371\177\000", __ss_align = 94701866020241}} con = 0x0 rxavp = 0x0 xname = {s = 0x0, len = -2141140176} __func__ = "via_builder" #1 0x000056217f155030 in create_via_hf (len=len@entry=0x7ffed7e3ecb8, msg=msg@entry=0x7ff9529548b0, send_info=send_info@entry=0x7ff764d20bb0, branch=branch@entry=0x7ffed7e3ecc0) at core/msg_translator.c:3232 via = <optimized out> extra_params = {s = 0x0, len = 0} hp = {host = 0x56217f4fb380 <default_global_address>, port = 0x56217f4fb370 <default_global_port>} sbuf = "\331\023\000\000\001", '\000' <repeats 18 times> --Type <RET> for more, q to quit, c to continue without paging--info locals slen = <optimized out> xparams = <optimized out> id_buf = <optimized out> id_len = 0 __func__ = "create_via_hf" __llevel = <optimized out> __kld = <optimized out> #2 0x000056217f1592f7 in build_req_buf_from_sip_req (msg=msg@entry=0x7ff9529548b0, returned_len=returned_len@entry=0x7ffed7e3eeac, send_info=send_info@entry=0x7ff764d20bb0, mode=mode@entry=128) at core/msg_translator.c:2096 len = 581 new_len = <optimized out> received_len = 0 rport_len = 0 uri_len = 0 via_len = 0 body_delta = 0 line_buf = 0x0 received_buf = 0x0 rport_buf = 0x0 new_buf = 0x0 buf = 0x56217f5866c0 <buf> "SUBSCRIBE sip:74@sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCa"... path_buf = {s = 0x0, len = 0} offset = 0 s_offset = 0 size = <optimized out> via_anchor = 0x7ff952951300 via_lump = <optimized out> via_rm = <optimized out> via_insert_param = 0x0 path_anchor = <optimized out> path_lump = <optimized out> branch = {s = 0x7ff952954ed8 "z9hG4bKc7ed.5819b4295fe83f981a2fc19b7f739d7e.0", len = 46} flags = 262273 --Type <RET> for more, q to quit, c to continue without paging--list udp_mtu = <optimized out> di = {send_sock = 0x0, to = {s = {sa_family = 18608, sa_data = "\225R\371\177\000\000\002\000\023\331\n4\001\r"}, sin = {sin_family = 18608, sin_port = 21141, sin_addr = { s_addr = 32761}, sin_zero = "\002\000\023\331\n4\001\r"}, sin6 = {sin6_family = 18608, sin6_port = 21141, sin6_flowinfo = 32761, sin6_addr = {__in6_u = { __u6_addr8 = "\002\000\023\331\n4\001\r\000\000\000\000\000\000\000", __u6_addr16 = {2, 55571, 13322, 3329, 0, 0, 0, 0}, __u6_addr32 = {3641901058, 218182666, 0, 0}}}, sin6_scope_id = 0}, sas = {ss_family = 18608, __ss_padding = "\225R\371\177\000\000\002\000\023\331\n4\001\r", '\000' <repeats 103 times>, __ss_align = 0}}, id = 0, send_flags = {f = 0, blst_imask = 0}, proto = 0 '\000', proto_pad0 = 0 '\000', proto_pad1 = 0} ret = <optimized out> __func__ = "build_req_buf_from_sip_req" error00 = <optimized out> #3 0x00007ff951fbe90d in prepare_new_uac (t=t@entry=0x7ff764d20880, i_req=i_req@entry=0x7ff9529548b0, branch=branch@entry=0, uri=<optimized out>, uri@entry=0x7ff9529548e8, path=<optimized out>, path@entry=0x7ff952954f60, next_hop=<optimized out>, fsocket=0x7ff9528f0e10, snd_flags=..., fproto=<optimized out>, flags=<optimized out>, instance=<optimized out>, ruid=<optimized out>, location_ua=<optimized out>) at ./src/modules/tm/t_fwd.c:482 shbuf = 0x0 add_rm_backup = <optimized out> body_lumps_backup = <optimized out> parsed_uri_bak = {user = { s = 0x56217f5866ce <buf+14> "74(a)sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCall-ID: 0_18056"..., len = 2}, passwd = {s = 0x0, len = 0}, host = { s = 0x56217f5866d1 <buf+17> "sip.domain.co SIP/2.0\r\nVia: SIP/2.0/UDP 12.171.207.82:25744;branch=z9hG4bK1805670803;rport\r\nFrom: \"Heather Myers\" <sip:1086448@sip.domain.co>;tag=1805518213\r\nTo: <sip:74@sip.domain.co>\r\nCall-ID: 0_18056078"..., len = 11}, port = {s = 0x0, len = 0}, params = {s = 0x0, len = 0}, sip_params = {s = 0x0, len = 0}, headers = { s = 0x0, len = 0}, port_no = 0, proto = 0, type = SIP_URI_T, flags = 0, transport = {s = 0x0, len = 0}, ttl = {s = 0x0, len = 0}, user_param = {s = 0x0, len = 0}, maddr = { s = 0x0, len = 0}, method = {s = 0x0, len = 0}, lr = {s = 0x0, len = 0}, r2 = {s = 0x0, len = 0}, gr = {s = 0x0, len = 0}, transport_val = {s = 0x0, len = 0}, ttl_val = { s = 0x0, len = 0}, user_param_val = {s = 0x0, len = 0}, maddr_val = {s = 0x0, len = 0}, method_val = {s = 0x0, len = 0}, lr_val = {s = 0x0, len = 0}, r2_val = {s = 0x0, len = 0}, gr_val = {s = 0x0, len = 0}} ret = -1 len = 32761 parsed_uri_ok_bak = <optimized out> free_new_uri = 1 msg_uri_bak = {s = 0x0, len = 0} dst_uri_bak = {s = 0x7ff9529525f0 "sip:10.52.1.13:5081", len = 19} dst_uri_backed_up = 1 path_bak = {s = 0x0, len = 0} free_path = 1 instance_bak = {s = 0x0, len = 0} ``` #### Log Messages  ``` ERROR 2025-01-10T06:00:31.847348563Z [resource.labels.containerName: kamailio] 8(28) CRITICAL: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [timer.h:188]: _set_fr_retr(): already added: 0x7ff763e62250 , tl=0x7ff763e62270!!! ERROR 2025-01-10T06:00:31.847388483Z [resource.labels.containerName: kamailio] 8(28) CRITICAL: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [t_fwd.c:1613]: t_send_branch(): BUG: retransmission already started for: 0x7ff763e62250 ERROR 2025-01-10T06:00:31.847394809Z [resource.labels.containerName: kamailio] 8(28) ERROR: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} sl [sl_funcs.c:428]: sl_reply_error(): stateless error reply used: No error (0/SL) ERROR 2025-01-10T06:00:31.847400738Z [resource.labels.containerName: kamailio] 8(28) BUG: {1 1 SUBSCRIBE 0_1907143137(a)192.168.100.28} tm [t_lookup.c:2052]: t_unref(): REQ_ERR DELAYED should have been caught much earlier for 0x7ff763e61f70: 24 (hex 18) ERROR 2025-01-10T06:00:47.346744773Z [resource.labels.containerName: kamailio] 45(65) ERROR: <core> [core/tcp_main.c:4818]: tcpconn_main_timeout(): connect 64.63.142.90:12190 failed (timeout) ERROR 2025-01-10T06:00:58.274915089Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 20 ERROR 2025-01-10T06:01:02.715789076Z [resource.labels.containerName: kamailio] 0(1) ALERT: <core> [main.c:805]: handle_sigs(): child process 26 exited by a signal 11 ERROR 2025-01-10T06:01:02.715827333Z [resource.labels.containerName: kamailio] 0(1) ALERT: <core> [main.c:809]: handle_sigs(): core was generated ERROR 2025-01-10T06:01:02.718815287Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 25 ERROR 2025-01-10T06:01:02.720985493Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 23 ERROR 2025-01-10T06:01:02.725450552Z [resource.labels.containerName: kamailio] 45(65) CRITICAL: <core> [core/pass_fd.c:281]: receive_fd(): EOF on 30 ``` #### SIP Traffic  ``` (paste your sip traffic here) ``` ### Possible Solutions I can atleast say that when this segfault happens that these logs are always present right before the crash. However, I also see instances of these logs without a crash immediately following. On the proxy we are doing this is withindlg route to send subscribes to the presence route: ``` if KSR.is_SUBSCRIBE() and KSR.is_myself_ruri() then -- in-dialog subscribe requests ksr_route_presence(); KSR.x.exit(); end ``` Any then this is our ksr_route_presence(): ``` function ksr_route_presence() if not KSR.is_SUBSCRIBE() then return 1; end local fuser = KSR.kx.get_fuser(); if fuser=='' then KSR.xlog.xinfo('Subscription attempt without username from IP ' .. KSR.kx.get_srcip() .. ' - Rejecting\n'); KSR.sl.sl_send_reply(404, 'Not Found'); KSR.x.exit(); end KSR.pv.sets('$avp(s:tenant)', string.match(fuser, '-(.*)')); local setId = '6' .. KSR.kx.get_def('DEFAULT_DISPATCH_SET'); local modeId = '7'; if KSR.dispatcher.ds_select_dst(setId, modeId)<0 then KSR.sl.send_reply(404, 'No destination'); KSR.x.exit(); end KSR.corex.set_send_socket(KSR.kx.get_def('PRIVATE_LISTEN_IP') .. ':' .. KSR.kx.get_def('INTERNAL_PORT')); KSR.tm.t_on_failure('ksr_route_rtf_dispatch'); ksr_route_relay(); end ``` ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 12.2.0 ``` * **Operating System**:  ``` PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" ``` ``` Linux gke-us-south1-external-sip-800ca69e-qyb3 5.15.0-1048-gke #53-Ubuntu SMP Tue Nov 28 00:39:01 UTC 2023 x86_64 GNU/Linux ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4102 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4102(a)github.com>

4 29

[kamailio/kamailio] [rtpengine] failure to ping rtpengine results in consistent core dump in kamailio 5.8.2 (Issue #4122)
by Brandon Armstead 26 Mar '25

26 Mar '25

### RTPEngine + RTIMER + SEGFAULT + unable to ping When unable to ping RTPEngine instances kamailio crashes. ### Troubleshooting Determined issue was related to the ping failures, resolved AWS SG issue, allowing successful pings. Issue subsequently went away. #### Reproduction Use rtpengine with RDS backend Load rtpengine with ~ 20 destinations you do not expect a response from on Use rtimer to call rtpengine.reload via jsonrpcs: ``` modparam("rtimer", "timer", "name=rtpengine;interval=60;mode=0;") modparam("rtimer", "exec", "timer=rtpengine;route=RELOAD_RTPENGINE") route[RELOAD_RTPENGINE] { xinfo("RTPEngine database reloading"); jsonrpc_exec('{"jsonrpc": "2.0", "method": "rtpengine.reload", "id": 1}'); } ``` #### Debugging Data Got a core dump -- although I deleted quickly (not thinking) as they were running instances out of disk quickly. bt full did not show much to step through all though clearly pointed to rtpengine. #### Log Messages ``` send_rtpp_command(): timeout waiting reply for command "ping" from RTPEngine <udp:10.0.X.X> ``` ### Possible Solutions TBD / do not have ping failures under heavy load. ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.8.2 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 10.2.1 ``` * **Operating System**: ``` uname -a Linux 5.10.0-27-cloud-amd64 #1 SMP Debian 5.10.205-2 (2023-12-31) x86_64 GNU/Linux cat /proc/version Linux version 5.10.0-27-cloud-amd64 (debian-kernel(a)lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.205-2 (2023-12-31) lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye ``` -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4122 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4122(a)github.com>

3 4

[kamailio/kamailio] Error when sending NOTIFYs in the presence of disconnected websocket watchers (Issue #4118)
by Flavio Grossi 25 Mar '25

25 Mar '25

### Description if multiple sip websocket clients are subscribed to some UA and one of the watchers suddenly disappears (e.g. the web page is reloaded), then no NOTIFY is ever sent after issuing a `pres_refresh_watchers()`. The following error is logged: ``` WARNING: <core> [core/msg_translator.c:3007]: via_builder(): TCP/TLS connection (id: 0) for WebSocket could not be found ERROR: tm [t_msgbuilder.c:1423]: assemble_via(): via building failed ERROR: tm [t_msgbuilder.c:1614]: build_uac_req(): error while assembling Via ERROR: tm [uac.c:552]: t_uac_prepare(): Error while building message ERROR: presence [notify.c:1734]: send_notify_request(): in function tm t_request_within() ERROR: presence [notify.c:1829]: notify(): sending Notify not successful ERROR: presence [notify.c:1585]: query_db_notify(): Could not send notify for [event]=dialog ERROR: presence [presence.c:739]: pres_refresh_watchers(): sending Notify requests ``` ### Troubleshooting when the client disconnects, the old watcher is not removed from the active_watchers table, instead it coexists with the new one, but the stale connection causes the error above when sending the notify, and then all of the following watchers (even the valid ones) are skipped too. Looking at the logs, the valid watchers are not notified because the while loop [here](https://github.com/kamailio/kamailio/blob/b96bc11a54f11ec688429c057bf… is skipped because of the `goto done` line in case of errors, which happens because it fails to build the via header for the message relevant to the stale websocket connection. By breaking out of the while loop, the remaining subs in the `subs_array` are skipped. #### Reproduction Just subscribe to a sip client using a websocket-based phone, then abruptly disconnect it (e.g. by reloading the web page) and then issue a `pres_refresh_watchers()` to make it generate a notify. #### Debugging Data #### Log Messages #### SIP Traffic ### Possible Solutions I tried to use the "websocket:closed" route to purge stale connections, but to my knowledge there are no APIs to clean invalid watchers. A possible solution could be to automatically remove the watchers on disconnection and maybe skip the invalid ones without exiting from the refresh loop ### Additional Information kamailio 5.8.2 * **Operating System**: Rocky Linux 9 -- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/4118 You are receiving this because you are subscribed to this thread. Message ID: <kamailio/kamailio/issues/4118(a)github.com>

3 4

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

sr-dev January 2025