### Description
Kamailio is crashing randomly 1 or 2 times a day. We have 2 kamailio with DMQ/Dialog sharing.
#### Debugging Data ``` #0 0x00007feb7466ddf3 in destroy_linkers () from /usr/lib64/kamailio/modules/dialog.so No symbol table info available. #1 0x00007feb7463a42e in dlg_dmq_handle_msg () from /usr/lib64/kamailio/modules/dialog.so No symbol table info available. #2 0x00007feb74d90718 in worker_loop () from /usr/lib64/kamailio/modules/dmq.so No symbol table info available. #3 0x00007feb74d73686 in ?? () from /usr/lib64/kamailio/modules/dmq.so No symbol table info available. #4 0x00000000005e79b0 in ?? () No symbol table info available. #5 0x00000000005e763c in ?? () No symbol table info available. #6 0x00000000005e763c in ?? () No symbol table info available. #7 0x00000000005e763c in ?? () No symbol table info available. #8 0x00000000005e763c in ?? () No symbol table info available. #9 0x00000000005e763c in ?? () No symbol table info available. #10 0x00000000005e763c in ?? () No symbol table info available. #11 0x00000000005e763c in ?? () No symbol table info available. #12 0x00000000005e80e9 in init_child () No symbol table info available. #13 0x0000000000426b94 in main_loop () ---Type <return> to continue, or q <return> to quit--- No symbol table info available. #14 0x000000000042e52a in main () No symbol table info available.
```
#### Log Messages
``` /usr/sbin/kamailio[9221]: CRITICAL: <core> [core/mem/q_malloc.c:521]: qm_free(): BUG: freeing already freed pointer (0x7feb781ee500), called from dialog: dlg_profile.c: destroy_linkers(291), first free dialog: dlg_profile.c: destroy_linkers(291) - ignoring
/usr/sbin/kamailio[9221]: ALERT: <core> [main.c:767]: handle_sigs(): child process 9495 exited by a signal 11 /usr/sbin/kamailio[9221]: ALERT: <core> [main.c:770]: handle_sigs(): core was generated
/usr/sbin/kamailio[9221]: CRITICAL: <core> [main.c:678]: sig_alarm_abort(): shutdown timeout triggered, dying... ``` ### Additional Information
``` version: kamailio 5.3.2 (x86_64/linux) 7ba545 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 7ba545 compiled on 11:29:01 Jan 9 2020 with gcc 4.4.7
```
* **Operating System**:
``` 2.6.32-042stab140.1 #1 SMP Thu Aug 15 13:32:22 MSK 2019 x86_64 x86_64 x86_64 GNU/Linux ```
Install the package with kamailio debug symbols and take again the gdb `bt full`. Otherwise the attached backtrace is not very useful.
bt full attached
We've run into a very similar issue running the 5.3.2 version on a Centos 7 system in or prod environment. If there's anything i can add that would help the investigation let me know.
This one went out of focus with the crisis out there, but if you still have the core dump, give the output for the following gdb commands:
``` frame 0 p *p_entry p *l p *lh frame 1 p *dlg p *msg ```
Besides providing the data requested in previous comment (which would be still useful for troubleshooting) try to run current master or 5.3 with the next commits backported:
* a4f8c0291bfd32752c0c2c323f618c6ac0c91d53 * 125031daeade9dab5c36b912dda65700d95c4c6a * 8965ff07f2e9fa105fc56896eed9caf4b6faf261
Commits backported to branch 5.3. Reopen if still an issue after testing.
Closed #2224.