23 Apr
2015
23 Apr
'15
10:26 a.m.
Hello,
I just got this crash:
(gdb) where
#0 0x00007fc3c1319625 in raise () from /lib64/libc.so.6
#1 0x00007fc3c131ae05 in abort () from /lib64/libc.so.6
#2 0x0000000000619d31 in fm_free (qm=0x7fc1b6ffa000, p=0x1e11a02400000000,
file=0x7fc3bf8ebbfd "tm: h_table.c", func=0x7fc3bf8ebed8
"free_cell",
line=162) at mem/f_malloc.c:588
#3 0x00007fc3bf8300a1 in free_cell (dead_cell=0x7fc1b71da540) at
h_table.c:162
#4 0x00007fc3bf831705 in free_hash_table () at h_table.c:448
#5 0x00007fc3bf85577a in tm_shutdown () at t_funcs.c:123
#6 0x0000000000590d79 in destroy_modules () at sr_module.c:811
#7 0x000000000049bb43 in cleanup (show_status=1) at main.c:569
#8 0x000000000049d10b in shutdown_children (sig=15, show_status=1)
at main.c:711
#9 0x000000000049f6e1 in handle_sigs () at main.c:802
#10 0x00000000004a6fbf in main_loop () at main.c:1757
#11 0x00000000004ab8bf in main (argc=11, argv=0x7fff1dbcda68) at main.c:2578
The fact that shutdown handlers were called makes me wonder if this is
not so much a crash as a foreseen condition that I am not properly
handling, like OOM. Can anyone shed any light?
Thanks,
--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States
Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
23 Apr
23 Apr
10:29 a.m.
So, my interpretation of this backtrace is that Kamailio simply crashed
during what would otherwise be ordinary unwinding of TM hash table
allocation. I suppose that means I should try to figure out why Kamailio
shut down in the first place, i.e. from the logging.
On 04/23/2015 03:26 AM, Alex Balashov wrote:
Hello,
I just got this crash:
(gdb) where
#0 0x00007fc3c1319625 in raise () from /lib64/libc.so.6
#1 0x00007fc3c131ae05 in abort () from /lib64/libc.so.6
#2 0x0000000000619d31 in fm_free (qm=0x7fc1b6ffa000, p=0x1e11a02400000000,
file=0x7fc3bf8ebbfd "tm: h_table.c", func=0x7fc3bf8ebed8
"free_cell",
line=162) at mem/f_malloc.c:588
#3 0x00007fc3bf8300a1 in free_cell (dead_cell=0x7fc1b71da540) at
h_table.c:162
#4 0x00007fc3bf831705 in free_hash_table () at h_table.c:448
#5 0x00007fc3bf85577a in tm_shutdown () at t_funcs.c:123
#6 0x0000000000590d79 in destroy_modules () at sr_module.c:811
#7 0x000000000049bb43 in cleanup (show_status=1) at main.c:569
#8 0x000000000049d10b in shutdown_children (sig=15, show_status=1)
at main.c:711
#9 0x000000000049f6e1 in handle_sigs () at main.c:802
#10 0x00000000004a6fbf in main_loop () at main.c:1757
#11 0x00000000004ab8bf in main (argc=11, argv=0x7fff1dbcda68) at
main.c:2578
The fact that shutdown handlers were called makes me wonder if this is
not so much a crash as a foreseen condition that I am not properly
handling, like OOM. Can anyone shed any light?
Thanks,
--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States
Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
10:31 a.m.
Ah, okay, I missed a separate core dump from another process, a minute
earlier, that I think reflects an actual aberrant condition:
rogram terminated with signal 11, Segmentation fault.
#0 0x00000000005fb62b in timer_list_expire (t=1749420105,
h=0x7fc1b7046af0,
slow_l=0x7fc1b70478f8, slow_mark=32972) at timer.c:877
877 _timer_rm_list(tl); /* detach */
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-lib-2.1.23-15.el6_6.1.x86_64 glibc-2.12-1.149.el6_6.5.x86_64
keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64
libcom_err-1.41.12-21.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64
nspr-4.10.6-1.el6_5.x86_64 nss-3.16.2.3-3.el6_6.x86_64
nss-softokn-freebl-3.14.3-22.el6_6.x86_64
nss-util-3.16.2.3-2.el6_6.x86_64 openldap-2.4.39-8.el6.x86_64
openssl-1.0.1e-30.el6_6.5.x86_64
postgresql92-libs-9.2.10-1PGDG.rhel6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) where
#0 0x00000000005fb62b in timer_list_expire (t=1749420105,
h=0x7fc1b7046af0,
slow_l=0x7fc1b70478f8, slow_mark=32972) at timer.c:877
#1 0x00000000005fbae3 in timer_handler () at timer.c:953
#2 0x00000000005fbf51 in timer_main () at timer.c:992
#3 0x00000000004a64ba in main_loop () at main.c:1700
#4 0x00000000004ab8bf in main (argc=11, argv=0x7fff1dbcda68) at main.c:2578
--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States
Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
10:42 a.m.
Hello,
so it was no restart of kamailio done by a tool/human at all, right?
Can you get from gdb:
p *tl
Cheers,
Daniel
On 23/04/15 09:31, Alex Balashov wrote:
Ah, okay, I missed a separate core dump from another
process, a minute
earlier, that I think reflects an actual aberrant condition:
rogram terminated with signal 11, Segmentation fault.
#0 0x00000000005fb62b in timer_list_expire (t=1749420105,
h=0x7fc1b7046af0,
slow_l=0x7fc1b70478f8, slow_mark=32972) at timer.c:877
877 _timer_rm_list(tl); /* detach */
Missing separate debuginfos, use: debuginfo-install
cyrus-sasl-lib-2.1.23-15.el6_6.1.x86_64
glibc-2.12-1.149.el6_6.5.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64
libselinux-2.0.94-5.8.el6.x86_64 nspr-4.10.6-1.el6_5.x86_64
nss-3.16.2.3-3.el6_6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64
nss-util-3.16.2.3-2.el6_6.x86_64 openldap-2.4.39-8.el6.x86_64
openssl-1.0.1e-30.el6_6.5.x86_64
postgresql92-libs-9.2.10-1PGDG.rhel6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) where
#0 0x00000000005fb62b in timer_list_expire (t=1749420105,
h=0x7fc1b7046af0,
slow_l=0x7fc1b70478f8, slow_mark=32972) at timer.c:877
#1 0x00000000005fbae3 in timer_handler () at timer.c:953
#2 0x00000000005fbf51 in timer_main () at timer.c:992
#3 0x00000000004a64ba in main_loop () at main.c:1700
#4 0x00000000004ab8bf in main (argc=11, argv=0x7fff1dbcda68) at
main.c:2578
--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, May 27-29, 2015
Berlin, Germany - http://www.kamailioworld.com
3508
days inactive
3508
days old
3 comments
2 participants
participants (2)
-
Alex Balashov -
Daniel-Constantin Mierla