sr-dev March 2021

sr-dev@lists.kamailio.org

28 participants
489 discussions

[kamailio/kamailio] dialog: not able to set the hangup_reason , when we hangup calls using xmlrpc. (#1737)

by Surendra Tiwari

### Description We are using xmlrpc to terminate the calls once customer balance reaches to zero. call got terminated but we will not be able to set the dialog variable related to hangup_reason. There should be a way during dlg.terminate_dlg xmlrpc request that if i mention the some parameter then it should set before sending bye to callee and caller.  ### Troubleshooting #### Reproduction  #### Debugging Data  ``` (paste your debugging data here) ``` #### Log Messages  ``` (paste your log messages here) ``` #### SIP Traffic  ``` (paste your sip traffic here) ``` ### Possible Solutions  ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` root@ip-172-31-4-160:/etc/kamailio# kamailio -version version: kamailio 5.1.4 (x86_64/linux) 6ebfda flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 6ebfda compiled on 11:48:26 Sep 7 2018 with gcc 4.8.4 ``` * **Operating System**:  ``` root@ip-172-31-4-160:/etc/kamailio# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 14.04.5 LTS Release: 14.04 Codename: trusty root@ip-172-31-4-160:/etc/kamailio# uname -a Linux ip-172-31-4-160 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1737

2 years, 2 months

[kamailio/kamailio] keep new features in a file in the repository (#668)

by Victor Seva

This will allow us to point https://www.kamailio.org/wiki/features/new-in-devel to the file and "force" developers to update that file with the new features when they are introduced in the repository Remaining developers to upgrade the wiki seems pointless and anyone can create a PR for changes --- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/668

2 years, 2 months

[kamailio/kamailio] kazoo: consuming of events fails if message has no body (#1837)

by mihovilkolaric

### Description Consuming messages without message body fails due to a JSON parsing error: ``` ERROR: kazoo [kz_json.c:287]: kz_json_parse(): Error parsing json: unexpected end of data ERROR: kazoo [kz_json.c:288]: kz_json_parse(): ERROR: kazoo [kz_amqp.c:2960]: kz_amqp_send_worker_event(): error parsing json body ``` This makes it unable to consume messages from RabbitMQ's event exchange plugin ( https://www.rabbitmq.com/event-exchange.html ), as the documentatin states: **"The message body is always blank."** ### Possible Solutions As a possible fix I tried following change in kz_amqp.c: ``` void kz_amqp_send_worker_event(kz_amqp_server_ptr server_ptr, amqp_envelope_t* envelope, kz_amqp_bind_ptr bind) { char buffer[100]; kz_amqp_cmd_ptr cmd = NULL; kz_amqp_consumer_delivery_ptr ptr = NULL; json_obj_ptr json_obj = NULL; json_object* JObj = NULL; str* message_id = NULL; int idx = envelope->channel-1; int worker = 0; int _kz_server_id = server_ptr->id; int msg_size = envelope->message.body.len; //begin of changes int routingkey_size = envelope->routing_key.len; if (msg_size == 0 && routingkey_size > 0 ) { char *routingkey_data = pkg_malloc(routingkey_size + 1); memset(routingkey_data, 0, routingkey_size + 1); memcpy(routingkey_data, (char*)envelope->routing_key.bytes, routingkey_size); LM_DBG("Message size: %d\n", msg_size); LM_DBG("Routing Key: %s\n", routingkey_data); char *pos = strchr(routingkey_data,'.'); while (pos){ *pos = '_'; pos = strchr(pos,'.'); } LM_DBG("New Routing Key: %s\n", routingkey_data); sprintf(buffer, "kazoo:%s", routingkey_data); if(kz_amqp_consumer_fire_event(buffer) != 0) { LM_ERR("kazoo:%s not found", routingkey_data); } pkg_free(routingkey_data); return; } char *json_data = pkg_malloc(msg_size + 1); if(!json_data) { LM_ERR("no more package memory available. needed %d\n", msg_size + 1); return; } char *json_data = pkg_malloc(msg_size + 1); //end of changes if(!json_data) { LM_ERR("no more package memory available. needed %d\n", msg_size + 1); return; } ``` The change is rather simple: If there is no message-body supplied, the event-route _kazoo:<binding_key>_ is called (after replacing any '.' with '_' in the routing_key). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/1837

2 years, 2 months

[kamailio/kamailio] kamctl - maintenance for dbtextdb.py tool (#2469)

by Daniel-Constantin Mierla

**kamctl** include a python tool that is parsing SQL statements to do operations over the **db_text** files, for the common db operations (e.g., insert, delete). It is located at: * https://github.com/kamailio/kamailio/tree/master/utils/kamctl/dbtextdb The tool was contributed long time ago and not kept up to date, being known not to work with Python3 (e.g., #2464). This issue is created to see if someone has the interest to pick up the maintenance of this tool, if not it will be removed from kamctl. Note that db_text works with text files, so it is not necessary to have a tool for managing the content of these files, they can be simply updated with any text editor. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2469

2 years, 5 months

[kamailio/kamailio] [kamctl] Dispatcher operations in a case of db_text are mostly incorrect (#2464)

by Igor Olhovskiy

### Description On creating dispatcher with `kamctl` in the case of `db_text` it fails ### Troubleshooting #### Reproduction Point `kamctlrc` to DB_TEXT. Run `kamctl dispatcher add 1 sip:asterisk_2:5060 1 3` Run 'kamctl dispatcher add 1 sip:asterisk_2:5060 1 3 'prefix=123' 'test1'' Run 'kamctl dispatcher show' Run 'kamctl dispatcher reload' #### Debugging Data ``` # kamctl dispatcher add 1 sip:asterisk_2:5060 1 3 attrs cannot be empty or null -e \E[37;31mERROR: dispatcher - SQL Error ``` ``` # kamctl dispatcher add 1 sip:asterisk_2:5060 1 3 'prefix=123' 'test1' Traceback (most recent call last): File "/usr/lib/x86_64-linux-gnu/kamailio//kamctl/dbtextdb/dbtextdb.py", line 1243, in <module> main(sys.argv) File "/usr/lib/x86_64-linux-gnu/kamailio//kamctl/dbtextdb/dbtextdb.py", line 1230, in main dataset = conn.Execute(' '.join(argv[1:])) File "/usr/lib/x86_64-linux-gnu/kamailio//kamctl/dbtextdb/dbtextdb.py", line 423, in Execute self.WriteTempTable() File "/usr/lib/x86_64-linux-gnu/kamailio//kamctl/dbtextdb/dbtextdb.py", line 1138, in WriteTempTable self.temp_file.write(header.strip() + '\n') File "/usr/lib/python3.7/tempfile.py", line 620, in func_wrapper return func(*args, **kwargs) TypeError: a bytes-like object is required, not 'str' -e \E[37;31mERROR: dispatcher - SQL Error ``` dump is showing active dispatchers, show - is not. ``` # kamctl dispatcher dump { "jsonrpc": "2.0", "result": { "NRSETS": 1, "RECORDS": [{ "SET": { "ID": 1, "TARGETS": [{ "DEST": { "URI": "sip:asterisk_2:5060", "FLAGS": "AP", "PRIORITY": 0 } }] } }] }, "id": 124 } root@e529c8cb4f62:/# kamctl dispatcher show -e dispatcher gateways ``` No result on reload ``` # kamctl dispatcher reload { "jsonrpc": "2.0", "result": { }, "id": 160 } ``` ### Possible Solutions I've tried to add data to the table manually, but here I got another issue. But on Kamailio load, the only first dispatcher is processed. Will add another issue on this. ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` /# kamailio -v version: kamailio 5.4.1 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 8.3.0 ``` * **Operating System**: Debian 10/Docker ``` # uname -a Linux 9b7efa852274 5.3.18-lp152.36-default #1 SMP Tue Aug 18 17:09:44 UTC 2020 (885251f) x86_64 GNU/Linux ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2464

2 years, 5 months

[kamailio/kamailio] crash related to functions in cfg file parser for invalid cfg (#2239)

by Henning Westerholt

Against current git master. ``` henning@static:~/repositories/kamailio$ cat etc/crash-4.cfg #!KAMAILIO loadmodule "sl.so" request_route { switch (1) { case -1: send_reply("404", "Not Found"); case -2: send_reply("405",("Method Not Allowed"); exit; } ``` > $ ./src/kamailio -f etc/crash-4.cfg -L src/modules/ -Y /tmp -E -d > 0(2278) CRITICAL: <core> [core/cfg.y:3549]: yyerror_at(): parse error in config file /home/henning/repositories/kamailio/etc/crash-4.cfg, line 10, column 43: syntax error > 0(2278) CRITICAL: <core> [core/cfg.y:3549]: yyerror_at(): parse error in config file /home/henning/repositories/kamailio/etc/crash-4.cfg, line 10, column 43: '('')' expected (function call) > Segmentation fault (core dumped) > henning@static:~/repositories/kamailio$ gdb src/kamailio core > Reading symbols from src/kamailio...done. > [New LWP 2278] > Core was generated by `./src/kamailio -f etc/crash-4.cfg -L src/modules/ -Y /tmp -E -d'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x00005580cdbd131e in append_action (a=0x7fad84db9300, b=0x7fad84db97f8) at core/route_struct.c:147 > 147 for(t=a; t->next; t=t->next); > (gdb) bt > #0 0x00005580cdbd131e in append_action (a=0x7fad84db9300, b=0x7fad84db97f8) at core/route_struct.c:147 > #1 0x00005580cdf9208b in yyparse () at core/cfg.y:2428 > #2 0x00005580cdb96c11 in main (argc=9, argv=0x7ffc2d9f6a68) at main.c:2309 > (gdb) f 0 > #0 0x00005580cdbd131e in append_action (a=0x7fad84db9300, b=0x7fad84db97f8) at core/route_struct.c:147 > 147 for(t=a; t->next; t=t->next); > (gdb) p a > $1 = (struct action *) 0x7fad84db9300 > (gdb) p b > $2 = (struct action *) 0x7fad84db97f8 > (gdb) p t > $3 = (struct action *) 0x80 > (gdb) p a->next > $4 = (struct action *) 0x80 > (gdb) p *t > Cannot access memory at address 0x80 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2239

2 years, 5 months

[kamailio/kamailio] [Feature Request] Add reload capability for geoip2 module (#2029)

by Joel Serrano

### Description Currently, the `geoip2` module loads the IP database at startup and keeps it in cache and there is no way to tell Kamailio to reload the database without a full restart. ### Possible Solutions To keep this module inline with other modules that use external datasources, I would suggest to implement a new RPC command and a new geoip2 function to reload the database defined in the `path` geoip2 modparam without the need of a restart. I'm not aware of the effort required for implementing either of them, but only having an RPC command would effectively solve this issue as you can call RPC commands from within Kamailio conf, thus, the reload function would not be something strictly required but more of a nice-to-have. ### Examples: Function: ``` geoip2_reload() ``` RPC command: ``` kamctl rpc gip2.reload / kamctl rpc geoip2.reload ``` Kamcmd command: ``` kamcmd geoip2.reload ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2029

2 years, 7 months

[kamailio/kamailio] register using uac module is getting send even if send_socket is not in this listening list of kamailio (#2624)

by Surendra Tiwari

### Description I am uac module with database. uacreg table have the send_socket column. i do have 2 kamailio server running. i want only kamailio-1 should send register to uacreg entries. for this i have used send_socket column with socket of kamailio-1. After this we can see both kamailio server sending registration and that's the problem.  ### Troubleshooting #### Reproduction loadmodule "uac.so" modparam("uac","auth_username_avp","$avp(auser)") modparam("uac","auth_password_avp","$avp(apass)") modparam("uac","auth_realm_avp","$avp(arealm)") modparam("uac", "reg_contact_addr", "EXTERNAL_IP:5060") modparam("uac", "reg_db_url", DBURL) and use send_socket into uacreg with kamailio-1 socket.  #### Debugging Data  ``` (paste your debugging data here) ``` #### Log Messages  ``` (paste your log messages here) ``` #### SIP Traffic  ``` (paste your sip traffic here) ``` ### Possible Solutions  ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.3.8 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 8.3.0 ``` * **Operating System**:  ``` root@ip-10-0-12-135:/etc/kamailio# uname -a Linux ip-10-0-12-135 5.3.0-1019-aws #21~18.04.1-Ubuntu SMP Mon May 11 12:33:03 UTC 2020 x86_64 GNU/Linux ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2624

2 years, 9 months

[kamailio/kamailio] server crashes every 10 hours (#2101)

by ekke56

### Description  ### Troubleshooting coredump data below. #### Reproduction  #### Debugging Data  ``` (gdb) bt full #0 0x000000000046ccdd in get_hash1_raw (s=0x5da51296 <Address 0x5da51296 out of bounds>, len=9) at core/hashes.h:109 p = 0x5da51296 <Address 0x5da51296 out of bounds> v = 4100353960 h = 0 #1 0x000000000046cfb6 in str_hash_get (ht=0xa6d6c0 <cnts_hash_table>, key=0x5da51296 <Address 0x5da51296 out of bounds>, len=9) at core/str_hash.h:94 h = 0 e = 0x19 #2 0x000000000046debe in cnt_hash_lookup (group=0x7ffc2db9d7c0, name=0x7eff79693b80) at core/counters.c:420 e = 0x7eff80c48c88 first = 0xbd8191bc2e950d55 cnt_rec = 0x7eff80c48cb0 #3 0x000000000046e752 in counter_lookup_str (handle=0x7ffc2db9d7d0, group=0x7ffc2db9d7c0, name=0x7eff79693b80) at core/counters.c:541 cnt_rec = 0x19 #4 0x00007eff7d65b447 in get_stat (name=0x7eff79693b80) at ../../core/counters.h:197 h = {id = 13} grp = {s = 0x0, len = 0} #5 0x00007eff7d65c113 in statsc_svalue (name=0x7eff79693b80, res=0x7eff79693f4b) at statsc_mod.c:149 stat = 0xd __FUNCTION__ = "statsc_svalue" #6 0x00007eff7d65db79 in statsc_timer (ticks=113205942, param=0x0) at statsc_mod.c:274 sm = 0x7eff79693b70 tn = 1571099286 n = 115 __FUNCTION__ = "statsc_timer" #7 0x00000000005db7f9 in sr_wtimer_exec (ticks=113205942, param=0x0) at core/timer_proc.c:390 wt = 0x7eff80c56f48 wn = 0x0 wp = 0x0 cs = 0 __FUNCTION__ = "sr_wtimer_exec" #8 0x00000000005daa29 in fork_sync_timer (child_id=-1, desc=0x7c8a71 "secondary timer", make_sock=1, f=0x5db4f2 <sr_wtimer_exec>, param=0x0, interval=1000) at core/timer_proc.c:224 pid = 0 ts1 = 1811295077 ts2 = 1000 #9 0x00000000005dbb26 in sr_wtimer_start () at core/timer_proc.c:416 __FUNCTION__ = "sr_wtimer_start" #10 0x0000000000425a33 in main_loop () at main.c:1729 i = 8 pid = 3615 si = 0x0 si_desc = "udp receiver child=7 sock=91.216.86.57:5060\000\377~\000\000@۹-\374\177\000\000\020\207A\000\000\000\000\000\020\336\271-\374\177\000\000\323c{\000\000\000\000\000\031\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\200\000\000\000\000\000__:\201\377~\000\000\250\346\260v\377~\000\000\270\314À\377~\000" nrprocs = 8 woneinit = 1 __FUNCTION__ = "main_loop" #11 0x000000000042c72a in main (argc=7, argv=0x7ffc2db9def8) at main.c:2696 cfg_stream = 0x1ee3010 c = -1 r = 0 tmp = 0x7ffc2db9ef2c "" tmp_len = 0 port = 0 proto = 0 options = 0x76a9a0 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 ---Type <return> to continue, or q <return> to quit--- info locals seed = 2620966285 rfd = 4 debug_save = 0 debug_flag = 0 dont_fork_cnt = 0 n_lst = 0x2 p = 0x0 st = {st_dev = 20, st_ino = 22346, st_nlink = 2, st_mode = 16832, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1568107728, tv_nsec = 458000000}, st_mtim = { tv_sec = 1571064422, tv_nsec = 57000000}, st_ctim = {tv_sec = 1571064422, tv_nsec = 57000000}, __unused = {0, 0, 0}} __FUNCTION__ = "main" (gdb) list 104 register unsigned v; 105 register unsigned h; 106 107 h=0; 108 109 hash_update_str(s, s+len, p, v, h); 110 return hash_finish(h); 111 } 112 113 ``` #### Log Messages  ``` no relevant messages ``` #### SIP Traffic  ``` seems to be triggered by internal timer ``` ### Possible Solutions  ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` version: kamailio 5.2.4 (x86_64/linux) 759867 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 759867 compiled on 12:20:22 Sep 11 2019 with gcc 4.8.5 ``` * **Operating System**:  ``` CentOS Linux release 7.4.1708 (Core) Linux xxx 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 21:24:32 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2101

2 years, 10 months

[kamailio/kamailio] Infinity read from closed socket (#2658)

by sergey-safarov

### Description I observing a case when Kamailio stop processing calls. What is found: *Lot of connections with CLOSE_WAIT state* ``` [root@sbc-13 ~]# netstat -anp | grep 5060 tcp 129 0 100.85.118.66:5060 0.0.0.0:* LISTEN 56907/kamailio tcp 153 0 100.85.118.66:5060 3.101.84.132:21929 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:32738 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:39084 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:28843 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:57939 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:27823 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:15820 CLOSE_WAIT - tcp 0 0 100.85.118.66:5060 207.244.110.239:11587 ESTABLISHED 56907/kamailio tcp 153 0 100.85.118.66:5060 3.101.84.132:11972 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:57245 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:54344 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:41416 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:3644 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:61657 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:49325 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:65334 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:48657 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:1612 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:7879 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:56626 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:57247 CLOSE_WAIT - tcp 153 0 100.85.118.66:5060 3.101.84.132:30494 CLOSE_WAIT - ``` At this time on two core server, one core loaded 100% and another core is free. And top show ``` %Cpu(s): 29.4 us, 26.5 sy ``` As about 26% CPU cycles proceed kernel call, using `strace` utility I found Kamailio process (with top CSP usage) continusly make thus kernel calls. ``` recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) recvfrom(58, 0x7ffdb46a5ab0, 232, MSG_PEEK, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable) ``` *Then I check 58 process file descriptor* ``` [root@sbc-13 ~]# ls -la /proc/56907/fd/58 lrwx------. 1 kamailio kamailio 64 Mar 3 10:49 /proc/56907/fd/58 -> 'socket:[3931924]' ``` *Info about open socket* ``` [root@sbc-13 ~]# lsof -a -p 56907| grep 3931924 kamailio 56907 kamailio 58u IPv4 3931924 0t0 TCP ip-100-85-118-66.us-west-1.compute.internal:sip->207.244.110.239:11495 (ESTABLISHED) ``` *Info about Kamailio 56907 process* ``` [root@sbc-13 ~]# kamcmd core.ps | tail 56903 tcp receiver (generic) child=4 56904 tcp receiver (generic) child=5 56905 tcp receiver (generic) child=6 56906 tcp receiver (generic) child=7 56907 tcp main process ``` Then using gdb I have created core file and back trace here ``` (gdb) bt full #0 0x00007f07d3639b1e in recv () from /lib64/libpthread.so.0 No symbol table info available. #1 0x0000000000528817 in tcpconn_read_haproxy (c=0x7f07ce98d930) at core/tcp_main.c:953 bytes = -1 retval = 0 size = 3454406656 port = 32519 p = 0x7ef068 "" end = 0x7f07d335c4c0 <syslog> "\363\017\036\372H\201\354", <incomplete sequence \330> src_ip = 0x6e9646 <qm_shm_malloc+103> dst_ip = 0x7ffdb46a5c20 v2sig = "\r\n\r\n\000\r\nQUIT\n" hdr = {v1 = {line = "\300\304\065\323\004", '\000' <repeats 11 times>, "\370ؘ\316\004\000\000\000t\251\205\000\000\000\000\000\320[j\264\375\177\000\000\004\331m\000\000\000\000\000\320[j\264\375\177\000\000\306\331m\000\000\000\000\000\b\342u\316\a\177\000\000\000\020\346\315\a\177\000\000\367b\201\000\000\000\000\000y\251\205\000n\004\000\000\260\235\201\000\000\000\000\000\247\227\201"}, v2 = {sig = "\300\304\065\323\004\000\000\000\000\000\000", ver_cmd = 0 '\000', fam = 0 '\000', len = 0, addr = {ip4 = {src_addr = 3466123512, dst_addr = 4, src_port = 43380, dst_port = 133}, ip6 = {src_addr = "\370ؘ\316\004\000\000\000t\251\205\000\000\000\000", dst_addr = "\320[j\264\375\177\000\000\004\331m\000\000\000\000", src_port = 23504, dst_port = 46186}, unx = {src_addr = "\370ؘ\316\004\000\000\000t\251\205\000\000\000\000\000\320[j\264\375\177\000\000\004\331m\000\000\000\000\000\320[j\264\375\177\000\000\306\331m\000\000\000\000\000\b\342u\316\a\177\000\000\000\020\346\315\a\177\000\000\367b\201\000\000\000\000\000y\251\205\000n\004\000\000\260\235\201\000\000\000\000\000\247\227\201\000\000\000\000\000\240C\000\000\000\000\000\000\000\020\346", <incomplete sequence \315>, dst_addr = "\a\177\000\000PB\226\316\a\177\000\000\000\020\346\315\a\177\000\000\274\332\366\312\a\177\000\000Ȧ\351͗\002\000\000\240\370\366\312\a\177\000\000S\342\366\312\a\177\000\000@\342u\316\a\177\000\000\000\000\000\000\000\000P\225\001\000\000\000\000\000\000\000\003\000\000\000\001\b\000\000\300[j\264\375\177\000\000R\031Q\000\000\000\000\000hV\313\322\a\177\000"}}}} __func__ = "tcpconn_read_haproxy" #2 0x0000000000529cc8 in tcpconn_new (sock=58, su=0x7ffdb46a5e00, local_addr=0x7f07d2cb5668, ba=0x7f07d2cb5610, type=2, state=3) at core/tcp_main.c:1164 c = 0x7f07ce98d930 rd_b_size = 16384 ret = 32519 __func__ = "tcpconn_new" #3 0x000000000055194e in handle_new_connect (si=0x7f07d2cb5610) at core/tcp_main.c:4259 su = {s = {sa_family = 2, sa_data = ",\347\317\364n\357\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 59180, sin_addr = {s_addr = 4017026255}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 59180, sin6_flowinfo = 4017026255, sin6_addr = {__in6_u = {__u6_addr8 = "\000\000\000\000\000\000\000\000h\360~\000\000\000\000", __u6_addr16 = {0, 0, 0, 0, 61544, 126, 0, 0}, __u6_addr32 = {0, 0, 8319080, 0}}}, sin6_scope_id = 67108864}, sas = {ss_family = 2, __ss_padding = ",\347\317\364n\357\000\000\000\000\000\000\000\000h\360~\000\000\000\000\000\000\000\000\004\000\000\000\000\360^j\264\375\177\000\000\023FU\000\000\000\000\000\230_\226\316\a\177\000\000\300\225\264\000\000\000\000\000\377\377\377\377\001\000\000\000\060٘\316\a\177\000\000\060٘\316\a\177\000\000\377\377\377\377\377\377\377\377\220^j\264\375\177\000\000\000\000\000\000\000\000\000\000\300\304\065\323\a\177\000", __ss_align = 8235966803625406752}} sock_name = {s = {sa_family = 23984, sa_data = "j\264\375\177\000\000\376\023Q\000\000\000\000"}, sin = {sin_family = 23984, sin_port = 46186, sin_addr = {s_addr = 32765}, sin_zero = "\376\023Q\000\000\000\000"}, sin6 = {sin6_family = 23984, sin6_port = 46186, sin6_flowinfo = 32765, sin6_addr = {__in6_u = {__u6_addr8 = "\376\023Q\000\000\000\000\000\310ژ\316\a\177\000", __u6_addr16 = {5118, 81, 0, 0, 56008, 52888, 32519, 0}, __u6_addr32 = {5313534, 0, 3466123976, 32519}}}, sin6_scope_id = 3454980136}, sas = {ss_family = 23984, __ss_padding = "j\264\375\177\000\000\376\023Q\000\000\000\000\000\310ژ\316\a\177\000\000(\320\356\315\a\177\000\000\030\"\202\000_\376\377\377\000\000\000\000\000\000\000\000\300\304\065\323\a\177\000\000\363:\204\000\000\000\000\000h\360~\000\000\000\000\000\000\000\000\004\000\000\000\000\360^j\264\375\177\000\000\245\222T\000\000\000\000\000\250\376\314\322\002\000\000\000\000\000\000\000\002\000\000\000Dc\201\000\241\001\000", __ss_align = 0}} sock_name_len = 32519 dst_su = 0x7f07d2cb5668 tcpconn = 0xffffffffce98d930 su_len = 16 new_sock = 58 __func__ = "handle_new_connect" #4 0x0000000000554aac in handle_io (fm=0x7f07d2ccfa40, ev=1, idx=-1) at core/tcp_main.c:4517 ret = 16 __func__ = "handle_io" #5 0x000000000051c992 in io_wait_loop_epoll (h=0xaf4760 <io_h>, t=5, repeat=0) at core/io_wait.h:1070 n = 1 r = 0 fm = 0x7f07d2ccfa40 revents = 1 __func__ = "io_wait_loop_epoll" #6 0x000000000055800f in tcp_main_loop () at core/tcp_main.c:4808 si = 0x0 r = 8 __func__ = "tcp_main_loop" #7 0x000000000042ca0b in main_loop () at main.c:1788 i = 8 pid = 0 si = 0x0 si_desc = "udp receiver child=7 sock=[2600:1f1c:6d5:aa03::7]:5060\000\061\063\062:5060)\000\000\000\000\060\000\000\000\370bj\264\375\177\000\000\020bj\264\375\177\000\000\000\334ѯ\344lFߨE\201\000\000\000\000\000\250\203\306\322k\000\000\000\363:\204\000\000\000\000\000\270o\305\322\a\177\000" nrprocs = 8 woneinit = 1 __func__ = "main_loop" #8 0x0000000000435d95 in main (argc=11, argv=0x7ffdb46a6828) at main.c:2895 cfg_stream = 0x156b2a0 c = -1 r = 0 tmp = 0x7ffdb46a6ee3 "" tmp_len = 0 port = 0 proto = 0 ahost = 0x0 aport = 0 options = 0x7f1e48 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:" ret = -1 seed = 2702569434 rfd = 4 debug_save = 0 debug_flag = 0 dont_fork_cnt = 2 n_lst = 0x7ffdb46d4268 p = 0x7f07d3309db5 <handle_intel.constprop+197> "H\205\300uf\213t$\004M\215\216\244" st = {st_dev = 24, st_ino = 19244, st_nlink = 2, st_mode = 16832, st_uid = 985, st_gid = 979, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1613817804, tv_nsec = 114998890}, st_mtim = {tv_sec = 1614768566, tv_nsec = 7667804}, st_ctim = {tv_sec = 1614768566, tv_nsec = 7667804}, __glibc_reserved = {0, 0, 0}} tbuf = '\000' <repeats 16 times>, "\214\344'\323\a\177\000\000\330Am\264\375\177\000\000\000\000\000\000 ", '\000' <repeats 19 times>, "X_\376\323\a\177\000\000\000ej\264\375\177\000\000\a\000\000\000\000\000\000\000\020U\376\323\a\177\000\000\a\000\000\000\t\000\000\000\210\351 \324\a\177\000\000\034\021\377\323\a\177\000\000\000\000\000\000\000\000\000\000H\037\377\323\a\177\000\000Pdj\264\375\177\000\000\260\344'\323\a\177\000\000\300\363\037\324\a\177\000\000\000\000\000\000\000\000\000\000\320dj\264\375\177", '\000' <repeats 26 times>, "\200W\376\323\a\177\000\000"... option_index = 12 long_options = {{name = 0x7f4056 "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x7ef4e1 "version", has_arg = 0, flag = 0x0, val = 118}, {name = 0x7f405b "alias", has_arg = 1, flag = 0x0, val = 1024}, {name = 0x7f4061 "subst", has_arg = 1, flag = 0x0, val = 1025}, {name = 0x7f4067 "substdef", has_arg = 1, flag = 0x0, val = 1026}, {name = 0x7f4070 "substdefs", has_arg = 1, flag = 0x0, val = 1027}, {name = 0x7f407a "server-id", has_arg = 1, flag = 0x0, val = 1028}, {name = 0x7f4084 "loadmodule", has_arg = 1, flag = 0x0, val = 1029}, {name = 0x7f408f "modparam", has_arg = 1, flag = 0x0, val = 1030}, {name = 0x7f4098 "log-engine", has_arg = 1, flag = 0x0, val = 1031}, {name = 0x7f40a3 "debug", has_arg = 1, flag = 0x0, val = 1032}, {name = 0x7f40a9 "cfg-print", has_arg = 0, flag = 0x0, val = 1033}, {name = 0x7f40b3 "atexit", has_arg = 1, flag = 0x0, val = 1034}, {name = 0x0, has_arg = 0, flag = 0x0, val = 0}} __func__ = "main" ``` We use enabled haproxy protocol. Loos as Kamailio does not properly close the socket when the socket closed on the client-side. ### Additional Information * **Kamailio Version** - output of `kamailio -v` ``` [root@sbc-13 ~]# kamailio -v version: kamailio 5.5.0-dev4 (x86_64/linux) 06656a flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 06656a compiled on 04:01:58 Mar 1 2021 with gcc 8.3.1 ``` * **Operating System**: ``` [root@sbc-13 ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="8" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="8" PLATFORM_ID="platform:el8" PRETTY_NAME="CentOS Linux 8" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:8" HOME_URL="https://centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-8" CENTOS_MANTISBT_PROJECT_VERSION="8" ``` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2658

2 years, 10 months

← Newer
1
2
3
4
5
6
7
8
9
...
49
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

sr-dev March 2021