17 Dec
2024
17 Dec
'24
7:48 a.m.
### Introduction
First, I apologize for the lack of detail here. I am unable to get a core dump and as a
production box, I was unable to run in debug mode.
### Description
Using Kamailio 5.8.4 on Debian 12, every 60-80 minutes, the software would crash, blaming
tls_wolfssl, such as:
```
2024-12-16T14:58:27.554497+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL:
<core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory
block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T14:58:27.554555+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9084]: CRITICAL:
<core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm.
0x7f3440ba7560 (address 0x7f3440ba75a0) beginning overwritten (23)! Memory allocator was
called from tls_wolfssl: tls_init.c:240. Fragment marked by :29. Exec from
core/mem/q_malloc.c:546.
2024-12-16T14:58:27.616558+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9089]: CRITICAL:
<core> [core/pass_fd.c:281]: receive_fd(): EOF on 103
2024-12-16T14:58:27.617684+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT:
<core> [main.c:805]: handle_sigs(): child process 9084 exited by a signal 6
2024-12-16T14:58:27.618339+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: ALERT:
<core> [main.c:809]: handle_sigs(): core was not generated
2024-12-16T14:58:27.618477+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[8992]: INFO:
<core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD
2024-12-16T14:58:27.618589+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[9088]: INFO:
<core> [main.c:888]: sig_usr(): signal 15 received
```
```
2024-12-16T14:57:52.029277+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL:
<core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x1 (out of memory
block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T14:57:52.029347+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21605]: CRITICAL:
<core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm.
0x7f085996b500 (address 0x7f085996b540) beginning overwritten (0)! Memory allocator was
called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from
core/mem/q_malloc.c:546.
2024-12-16T14:57:52.048597+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21611]: CRITICAL:
<core> [core/pass_fd.c:281]: receive_fd(): EOF on 101
2024-12-16T14:57:52.048921+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT:
<core> [main.c:805]: handle_sigs(): child process 21605 exited by a signal 6
2024-12-16T14:57:52.048953+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: ALERT:
<core> [main.c:809]: handle_sigs(): core was not generated
2024-12-16T14:57:52.048975+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21512]: INFO:
<core> [main.c:832]: handle_sigs(): terminating due to SIGCHLD
2024-12-16T14:57:52.048997+00:00 ip-172-16-0-73 /usr/local/sbin/kamailio[21609]: INFO:
<core> [main.c:888]: sig_usr(): signal 15 received
```
```
2024-12-16T16:21:35.967576+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[10591]: CRITICAL:
<core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm.
0x7f442778d110 (address 0x7f442778d150) beginning overwritten (0)! Memory allocator was
called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from
core/mem/q_malloc.c:546.
```
```
2024-12-16T13:31:55.995222+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7602]: CRITICAL:
<core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer
(0x7f8e55830a70), called from tls_wolfssl: tls_init.c: ser_free(240), first free
tls_wolfssl: tls_init.c: ser_malloc(228) - ignoring
2024-12-16T13:32:40.093954+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL:
<core> [core/mem/q_malloc.c:535]: qm_free(): BUG: bad pointer 0x7a100000000 (out of
memory block!) called from tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T13:32:40.094047+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7603]: CRITICAL:
<core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm.
0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was
called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from
core/mem/q_malloc.c:546.
2024-12-16T13:32:40.260913+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL:
<core> [core/mem/q_malloc.c:555]: qm_free(): BUG: freeing already freed pointer
(0x7f8e55761a70), called from core: core/tcp_main.c: _tcpconn_free(1666), first free
tls_wolfssl: tls_init.c: ser_free(240) - ignoring
2024-12-16T13:32:40.263520+00:00 ip-172-16-0-52 /usr/local/sbin/kamailio[7504]: CRITICAL:
<core> [core/mem/q_malloc.c:126]: qm_debug_check_frag(): BUG: qm: fragm.
0x7f8e55762720 (address 0x7f8e55762760) beginning overwritten (0)! Memory allocator was
called from tls_wolfssl: tls_init.c:240. Fragment marked by (null):0. Exec from
core/mem/q_malloc.c:546.
```
The system was handling ~900 TLS registrations with ~2700 presence SUBSCRIBE events.
**Running the same config with `tls.so` does not result in crash.**
### Additional Information
* **Kamailio Version** - 5.8.4
```
version: kamailio 5.8.4 (x86_64/linux) 598105-dirty
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST,
DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_SEND_BUFFER_SIZE 262144,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 598105 -dirty
compiled on 04:31:34 Dec 16 2024 with gcc 12.2.0
```
* **Operating System**:
```
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
Linux ip-172-16-0-52 6.1.0-28-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1
(2024-11-22) x86_64 GNU/Linux
```
* TCP Settings
```
children=16
disable_tcp=no
tcp_connection_lifetime=3605
tcp_accept_no_cl=yes
tcp_max_connections=8192
tcp_defer_accept=yes
tcp_accept_unique=0
tcp_connection_lifetime=3605
tcp_connection_match=1
tcp_connect_timeout=10
tcp_crlf_ping=yes
tcp_linger2=1
tcp_keepalive=yes
tcp_reuse_port=yes
tcp_keepidle=20
tcp_keepintvl=15
tcp_rd_buf_size=65536
tcp_msg_data_timeout=40
tcp_msg_read_timeout=40
tcp_check_timer=20
tls_threads_mode=2
tls_max_connections=8192
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/4074
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/4074(a)github.com>