29 Jun
2020
29 Jun
'20
1:34 p.m.
<!--
Kamailio Project uses GitHub Issues only for bugs in the code or feature requests. Please
use this template only for bug reports.
If you have questions about using Kamailio or related to its configuration file, ask on
sr-users mailing list:
* http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
If you have questions about developing extensions to Kamailio or its existing C code, ask
on sr-dev mailing list:
* http://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
Please try to fill this template as much as possible for any issue. It helps the
developers to troubleshoot the issue.
If there is no content to be filled in a section, the entire section can be removed.
You can delete the comments from the template sections when filling.
You can delete next line and everything above before submitting (it is a comment).
-->
### Description
Hi, I ran into an issue where the Kamailio service seemingly froze (didn't crash, but
failed to receive and deliver calls) while running, I went through the logs a little bit,
and saw the same log pattern (pasted below) recouring every few months or so, on an older
installation (5.2.0). Usually, it ended up automatically restarting the service. But not
this time, I had to reload it manually to make the service work again.
Is this a known issue?
How can I make sure that the service will be able to restart the next time it happens?
Edward
### Troubleshooting
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
```
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/local/sbin/kamailio...done.
warning: exec file is newer than core file.
[New LWP 11413]
Core was generated by `kamailio'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f4e4a52e428 in ?? ()
(gdb) Quit
(gdb) bt full
#0 0x00007f4e4a52e428 in ?? ()
No symbol table info available.
#1 0x00007f4e4a53002a in ?? ()
No symbol table info available.
#2 0x0000000000000020 in ?? ()
No symbol table info available.
#3 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) info locals
No symbol table info available.
(gdb) list
1871 & now if we don't need it */
1872 #ifdef USE_SLOW_TIMER
1873 + 1 /* slow timer process */
1874 #endif
1875 #ifdef USE_TCP
1876 +((!tcp_disable)?( 1/* tcp main */ + tcp_listeners ):0)
1877 #endif
1878 #ifdef USE_SCTP
1879 +((!sctp_disable)?sctp_listeners:0)
1880 #endif
(gdb)
```
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them
next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
```
Jun 28 16:35:22 kamprodegres kamailio: INFO: <core> [core/sctp_core.c:74]:
sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
Jun 28 16:35:22 kamprodegres kamailio: WARNING: <core> [core/socket_info.c:1394]:
fix_hostname(): could not rev. resolve 70.36.25.87
Jun 28 16:35:22 kamprodegres kamailio: WARNING: <core> [core/socket_info.c:1394]:
fix_hostname(): could not rev. resolve 70.36.25.87
Jun 28 16:35:22 kamprodegres kamailio: INFO: <core> [core/tcp_main.c:5042]:
init_tcp(): using epoll_lt as the io watch method (auto detected)
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: jsonrpcs [jsonrpcs_sock.c:197]:
jsonrpc_dgram_mod_init(): the socket /var/run/kamailio/kamailio_rpc.sock already exists,
trying to delete it...
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: rr [../outbound/api.h:52]:
ob_load_api(): unable to import bind_ob - maybe module is not loaded
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: rr [rr_mod.c:177]: mod_init(): outbound
module not available
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [main.c:2779]: main():
processes (at least): 40 - shm size: 67108864 - pkg size: 8388608
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 212992
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 425984
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 212992
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 425984
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:154]:
probe_max_receive_buffer(): SO_RCVBUF is initially 212992
Jun 28 16:35:22 kamprodegres kamailio[8489]: INFO: <core> [core/udp_server.c:206]:
probe_max_receive_buffer(): SO_RCVBUF is finally 425984
Jun 28 16:35:22 kamprodegres kamailio[8518]: INFO: jsonrpcs [jsonrpcs_sock.c:443]:
jsonrpc_dgram_process(): a new child 0/8518
Jun 28 16:35:22 kamprodegres kamailio[8519]: INFO: ctl [io_listener.c:214]:
io_listen_loop(): io_listen_loop: using epoll_lt io watch method (config)
Jun 29 18:13:37 kamprodegres kamailio[8513]: INFO: {1 581461 CANCEL
402954192_49936786(a)67.231.13.146} tm [t_reply.c:478]: _reply_light(): can't generate
487 reply when a final 404 was sent out
Jun 29 18:27:00 kamprodegres kamailio[8508]: INFO: {1 211236 CANCEL
405019408_127896329(a)67.231.13.146} tm [t_reply.c:478]: _reply_light(): can't generate
487 reply when a final 404 was sent out
Jun 29 18:51:10 kamprodegres /usr/local/sbin/kamailio[2269]: NOTICE: <core>
[main.c:725]: handle_sigs(): Thank you for flying kamailio!!!
Jun 29 18:51:10 kamprodegres /usr/local/sbin/kamailio[2326]: INFO: <core>
[main.c:847]: sig_usr(): signal 15 received
```
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.3.2 (x86_64/linux) 7ba545
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST,
DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY,
USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR,
USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535,
DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 7ba545
compiled on 16:37:32 Jun 14 2020 with gcc 5.4.0
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04,
CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->
```
Ubuntu 16.04
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380
29 Jun
29 Jun
3:59 p.m.
Hello, you need to install kamailio debugging symbols to get meaningful information from
the backtrace. From the log message it looks ok. You should first try to update to the
latest 5.3 version (e.g. 5.3.5 released recently), as it might be already fixed.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651328024
4:03 p.m.
Hello, you need to install kamailio debugging symbols
to get meaningful information from the backtrace. From the log message it looks ok. You
should first try to update to the latest 5.3 version (e.g. 5.3.5 released recently), as it
might be already fixed.
Can I do that after the coredump has already been generated?
If so, can you point me to how can I do that?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651329820
4:05 p.m.
Yes, look for a packaged called kamailio-dbg, kamailio-debug or similar and install it.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651330503
4:22 p.m.
Hmm. I did install it, but I get the same output.
`root@kamprodegres:/# gdb /usr/local/sbin/kamailio core
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/local/sbin/kamailio...done.
warning: exec file is newer than core file.
[New LWP 11413]
Core was generated by `kamailio'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f4e4a52e428 in ?? ()
(gdb) bt full
#0 0x00007f4e4a52e428 in ?? ()
No symbol table info available.
#1 0x00007f4e4a53002a in ?? ()
No symbol table info available.
#2 0x0000000000000020 in ?? ()
No symbol table info available.
#3 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) info locals
No symbol table info available.
(gdb) Quit
(gdb) quit
root@kamprodegres:/# apt-get install kamailio-dbg
Reading package lists... Done
Building dependency tree
Reading state information... Done
kamailio-dbg is already the newest version (4.3.4-1.1ubuntu2.1).
The following packages were automatically installed and are no longer required:
gyp javascript-common libjs-inherits libjs-jquery libjs-node-uuid libjs-underscore
libuv1 libuv1-dev node-abbrev node-ansi node-ansi-color-table node-archy node-async
node-block-stream node-combined-stream node-cookie-jar
node-delayed-stream node-forever-agent node-form-data node-fstream node-fstream-ignore
node-github-url-from-git node-glob node-graceful-fs node-gyp node-inherits node-ini
node-json-stringify-safe node-lockfile node-lru-cache
node-mime node-minimatch node-mkdirp node-mute-stream node-node-uuid node-nopt
node-normalize-package-data node-npmlog node-once node-osenv node-qs node-read
node-read-package-json node-request node-retry node-rimraf node-semver
node-sha node-sigmund node-slide node-tar node-tunnel-agent node-underscore node-which
python-pkg-resources
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 442 not upgraded.
root@kamprodegres:/#
`
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651339931
4:24 p.m.
Ok, in this case it the backtrace information might be from somewhere else. But I would
suggest to update to the lastest kamailio version first. It looks a bit like a crash
during shutdown, which might be already fixed.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651341362
30 Jun
30 Jun
1:27 a.m.
Are you aware of similar bug that has been fixed?
Edward
On Mon, Jun 29, 2020, 23:24 Henning Westerholt <notifications(a)github.com>
wrote:
Ok, in this case it the backtrace information might be
from somewhere
else. But I would suggest to update to the lastest kamailio version first.
It looks a bit like a crash during shutdown, which might be already fixed.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/kamailio/kamailio/issues/2380#issuecomment-651341362>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACM76S472YM6GDETFOBEX4LRZD2A5ANCNFSM4OLN6KAQ>
.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651547173
2:01 a.m.
This is hard to say without more details. You'll find all fixes in the Changelog:
https://www.kamailio.org/pub/kamailio/5.3.5/ChangeLog
Close this one, please re-open if the bug appears again after you updated to 5.3.5.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-651560460
2:01 a.m.
Closed #2380.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#event-3495969512
27 Jun
27 Jun
2:26 a.m.
I have the same issue in Kamailio 5.3.5, right now. Here is backtrace information.
warning: exec file is newer than core file.
[New LWP 56381]
Core was generated by `/usr/local/sbin/kamailio -P /run/kamailio/kamailio.pid -f
/usr/local/etc/kamail'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f7837e917bb in ?? ()
(gdb) bt full
#0 0x00007f7837e917bb in ?? ()
No symbol table info available.
#1 0x0000000000002000 in ?? ()
No symbol table info available.
#2 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) info locals
No symbol table info available.
(gdb)
No symbol table info available.
(gdb)
No symbol table info available.
(gdb)
No symbol table info available.
(gdb)
No symbol table info available.
(gdb)
No symbol table info available.
(gdb) list
1883 +((!sctp_disable)?sctp_listeners:0)
1884 #endif
1885 ;
1886 }
1887
1888 int main(int argc, char** argv)
1889 {
1890
1891 FILE* cfg_stream;
1892 int c,r;
(gdb)
I know i am using old version of Kamailio (5.3.5), But i want to know what is the problem?
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-2193897312
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/2380/2193897312(a)github.com>
2:29 a.m.
As already discussed, its unfortunately not possible to support this old end of life
versions in the open source project. Please upgrade to a supported version (5.7.x or
5.8.x) of Kamailio.
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/2380#issuecomment-2193900892
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/2380/2193900892(a)github.com>
48
days inactive
1507
days old
10 comments
3 participants
participants (3)
-
Henning Westerholt -
Mojtaba Esfandiari.S
-
ObakeFilter