So the first of those crashes after setting tls init_mode 1 wrote this to the logs

Sep 14 02:13:40 kamrtc01 /usr/sbin/kamailio[8815]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8603d45dc8), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:13:40 kamrtc01 /usr/sbin/kamailio[8813]: CRITICAL: <core> [core/mem/q_malloc.c:123]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8603d45e10 (address 0x7f8603d45e48) beginning overwritten (195)! Memory allocator was called from tls: tls_init.c:405. Fragment marked by :140213825900370. Exec from core/mem/q_malloc.c:511.
Sep 14 02:13:40 kamrtc01 systemd[1]: Started Process Core Dump (PID 9454/UID 0).
Sep 14 02:13:41 kamrtc01 systemd-coredump[9455]: Process 8813 (kamailio) of user 990 dumped core.#012#012Stack trace of thread 8813:#012#0  0x00007f86152a158c __pthread_kill_implementation (libc.so.6 + 0xa158c)#012#1  0x00007f8615254d46 raise (libc.so.6 + 0x54d46)#012#2  0x00007f86152287f3 abort (libc.so.6 + 0x287f3)#012#3  0x00000000007164db qm_debug_check_frag (kamailio + 0x3164db)#012#4  0x000000000071a703 qm_free (kamailio + 0x31a703)#012#5  0x0000000000725209 qm_shm_free (kamailio + 0x325209)#012#6  0x00007f861344a3d6 ser_free (tls.so + 0x313d6)#012#7  0x00007f8613d6d737 ERR_pop_to_mark (libcrypto.so.3 + 0x16d737)#012#8  0x00007f8613db3e42 ossl_namemap_doall_names (libcrypto.so.3 + 0x1b3e42)#012#9  0x00007f8613d9f225 EVP_PKEY_set_type_by_keymgmt (libcrypto.so.3 + 0x19f225)#012#10 0x00007f8613d9859a evp_keymgmt_util_assign_pkey (libcrypto.so.3 + 0x19859a)#012#11 0x00007f8613da21c3 EVP_PKEY_generate (libcrypto.so.3 + 0x1a21c3)#012#12 0x00007f861418082d ssl_generate_param_group (libssl.so.3 + 0x2682d)#012#13 0x00007f86141b52ea tls_parse_ctos_key_share (libssl.so.3 + 0x5b2ea)#012#14 0x00007f86141a7f79 tls_parse_all_extensions (libssl.so.3 + 0x4df79)#012#15 0x00007f86141d1cde ossl_statem_server_post_process_message (libssl.so.3 + 0x77cde)#012#16 0x00007f86141bc2cb state_machine.part.0 (libssl.so.3 + 0x622cb)#012#17 0x00007f8613472584 tls_accept (tls.so + 0x59584)#012#18 0x00007f861347d8be tls_h_read_f (tls.so + 0x648be)#012#19 0x00000000006c79bc tcp_read_headers (kamailio + 0x2c79bc)#012#20 0x00000000006cfb94 tcp_read_req (kamailio + 0x2cfb94)#012#21 0x00000000006d4cc3 handle_io (kamailio + 0x2d4cc3)#012#22 0x00000000006c271d io_wait_loop_epoll (kamailio + 0x2c271d)#012#23 0x00000000006d7a47 tcp_receive_loop (kamailio + 0x2d7a47)#012#24 0x00000000006b7689 tcp_init_children (kamailio + 0x2b7689)#012#25 0x000000000042fae9 main_loop (kamailio + 0x2fae9)#012#26 0x0000000000439969 main (kamailio + 0x39969)#012#27 0x00007f861523feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)#012#28 0x00007f861523ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)#012#29 0x000000000041dc65 _start (kamailio + 0x1dc65)#012ELF object binary architecture: AMD x86-64

and the second wrote

Sep 14 02:14:53 kamrtc01 /usr/sbin/kamailio[9487]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f931d685ac8), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:14:53 kamrtc01 kernel: kamailio[9487]: segfault at 4 ip 00007f932d339fa4 sp 00007ffd5e317f58 error 4 in libcrypto.so.3.0.7[7f932d2ad000+25b000]
Sep 14 02:14:53 kamrtc01 kernel: Code: 00 00 00 0f 1f 00 f3 0f 1e fa 8b 47 20 c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 8b 47 40 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa <48> 8b 07 8b 40 04 c3 0f 1f 44 00 00 f3 0f 1e fa 89 77 24 c3 0f 1f

Sep 14 02:14:53 kamrtc01 systemd-coredump[9542]: Process 9487 (kamailio) of user 990 dumped core.#012#012Stack trace of thread 9487:#012#0  0x00007f932d339fa4 EC_GROUP_get_field_type (libcrypto.so.3 + 0x139fa4)#012#1  0x00007f932d33cb59 ossl_ec_key_public_check_quick (libcrypto.so.3 + 0x13cb59)#012#2  0x00007f932d46fb15 ec_validate.lto_priv.0 (libcrypto.so.3 + 0x26fb15)#012#3  0x00007f932d3a1ca1 try_provided_check (libcrypto.so.3 + 0x1a1ca1)#012#4  0x00007f932d3a1d31 evp_pkey_public_check_combined (libcrypto.so.3 + 0x1a1d31)#012#5  0x00007f932d397943 EVP_PKEY_derive_set_peer_ex (libcrypto.so.3 + 0x197943)#012#6  0x00007f932df81b6b ssl_derive (libssl.so.3 + 0x28b6b)#012#7  0x00007f932dfce63b ossl_statem_server_process_message (libssl.so.3 + 0x7563b)#012#8  0x00007f932dfbadbf state_machine.part.0 (libssl.so.3 + 0x61dbf)#012#9  0x00007f932cb24584 tls_accept (tls.so + 0x59584)#012#10 0x00007f932cb2f8be tls_h_read_f (tls.so + 0x648be)#012#11 0x00000000006c79bc tcp_read_headers (kamailio + 0x2c79bc)#012#12 0x00000000006cfb94 tcp_read_req (kamailio + 0x2cfb94)#012#13 0x00000000006d6097 handle_io (kamailio + 0x2d6097)#012#14 0x00000000006c271d io_wait_loop_epoll (kamailio + 0x2c271d)#012#15 0x00000000006d7a47 tcp_receive_loop (kamailio + 0x2d7a47)#012#16 0x00000000006b7689 tcp_init_children (kamailio + 0x2b7689)#012#17 0x000000000042fae9 main_loop (kamailio + 0x2fae9)#012#18 0x0000000000439969 main (kamailio + 0x39969)#012#19 0x00007f932e83feb0 __libc_start_call_main (libc.so.6 + 0x3feb0)#012#20 0x00007f932e83ff60 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3ff60)#012#21 0x000000000041dc65 _start (kamailio + 0x1dc65)#012ELF object binary architecture: AMD x86-64
Sep 14 02:14:53 kamrtc01 systemd[1]: systemd-coredump@9-9541-0.service: Deactivated successfully.
Sep 14 02:14:53 kamrtc01 /usr/sbin/kamailio[9488]: CRITICAL: <core> [core/pass_fd.c:84]: recv_all(): 1st recv on 46 failed: Connection reset by peer
Sep 14 02:14:53 kamrtc01 /usr/sbin/kamailio[9488]: CRITICAL: <core> [core/tcp_main.c:3635]: handle_tcp_child(): read from tcp child 3 (pid 9487, no 25) Connection reset by peer [104]
Sep 14 02:14:53 kamrtc01 /usr/sbin/kamailio[9488]: CRITICAL: <core> [core/pass_fd.c:277]: receive_fd(): EOF on 43

I've also seen that in the current run that has not crashed I see things like this in the log

Sep 14 02:26:37 kamrtc01 /usr/sbin/kamailio[9593]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6efd0078), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:26:37 kamrtc01 /usr/sbin/kamailio[9594]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6eccf120), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:28:47 kamrtc01 /usr/sbin/kamailio[9591]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6ecf01a8), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:28:47 kamrtc01 /usr/sbin/kamailio[9594]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6efd72c8), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:32:09 kamrtc01 /usr/sbin/kamailio[9592]: INFO: <script>: START: REGISTER from sip:webrtc@webrtc.nebulaip.com (IP:92.239.255.130:50125)
Sep 14 02:32:09 kamrtc01 /usr/sbin/kamailio[9592]: INFO: <script>: START: REGISTER from sip:webrtc@webrtc.nebulaip.com (IP:92.239.255.130:50125)
Sep 14 02:32:12 kamrtc01 /usr/sbin/kamailio[9593]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6ef7f298), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring
Sep 14 02:32:12 kamrtc01 /usr/sbin/kamailio[9594]: CRITICAL: <core> [core/mem/q_malloc.c:519]: qm_free(): BUG: freeing already freed pointer (0x7f8f6ef7f3a0), called from tls: tls_init.c: ser_free(405), first free tls: tls_init.c: ser_free(405) - ignoring

I think that in every case of those sets of pairs of messages they are associated with those START:REGISTER from messages that come from the same ip address and same port with the same timestamp.


Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: <kamailio/kamailio/issues/3563/1718692496@github.com>