### Description ``` Kamailio crashes out with message: CRITICAL: <core> [pass_fd.c:277]: receive_fd(): EOF on 30 ```
### Troubleshooting
#### Reproduction
Happens sporadicly. Some times 6 times in a day. Some times it won't happen for a week. But on average it probably happens once per day.
#### Debugging Data
``` Reading symbols from /usr/local/sbin/kamailio...done. [New LWP 29579] [New LWP 9469] [New LWP 29581] Missing separate debuginfo for /usr/lib64/libpq.so.5 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/00/595f0be1374285c79ea1d4cdcee72682108fab.debug [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Missing separate debuginfo for /usr/lib64/libstdc++.so.6 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/43/4c339faa62ca4e59eb71572a60967fe54a69ad.debug Missing separate debuginfo for /lib64/libgcc_s.so.1 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/3f/d5f89de59e124ab1419a0bd16775b4096e84fd.debug Core was generated by `/usr/local/sbin/kamailio -P /var/run/kamailio.pid -m 256 -M 8 -u kamailio -g ka'. Program terminated with signal 11, Segmentation fault. #0 0x00007fc9a9350521 in _int_malloc () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.23-13.16.amzn1.x86_64 glibc-2.17-106.168.amzn1.x86_64 keyutils-libs-1.5.8-3.12.amzn1.x86_64 krb5-libs-1.13.2-12.40.amzn1.x86_64 libcom_err-1.42.12-4.40.amzn1.x86_64 libcurl-7.47.1-9.68.amzn1.x86_64 libevent-2.0.21-4.19.amzn1.x86_64 libicu-50.1.2-11.12.amzn1.x86_64 libidn-1.18-2.8.amzn1.x86_64 libpsl-0.6.2-1.2.amzn1.x86_64 libselinux-2.1.10-3.22.amzn1.x86_64 libssh2-1.4.2-2.13.amzn1.x86_64 libxml2-2.9.1-6.3.49.amzn1.x86_64 nspr-4.11.0-1.37.amzn1.x86_64 nss-3.21.3-2.77.amzn1.x86_64 nss-softokn-freebl-3.16.2.3-14.4.39.amzn1.x86_64 nss-util-3.21.3-1.1.51.amzn1.x86_64 openldap-2.4.40-12.29.amzn1.x86_64 openssl-1.0.1k-15.96.amzn1.x86_64 xz-libs-5.1.2-12alpha.12.amzn1.x86_64 zlib-1.2.8-7.18.amzn1.x86_64
(gdb) bt full #0 0x00007fc9a9350521 in _int_malloc () from /lib64/libc.so.6 No symbol table info available. #1 0x00007fc9a935226c in malloc () from /lib64/libc.so.6 No symbol table info available. #2 0x00007fc9a9ac45d5 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #3 0x00007fc9a9abf691 in _dl_map_object_deps () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #4 0x00007fc9a9ac589b in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #5 0x00007fc9a9ac11b4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #6 0x00007fc9a9ac51ab in _dl_open () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #7 0x00007fc9a94034d2 in do_dlopen () from /lib64/libc.so.6 No symbol table info available. #8 0x00007fc9a9ac11b4 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #9 0x00007fc9a9403592 in __libc_dlopen_mode () from /lib64/libc.so.6 No symbol table info available. #10 0x00007fc9a93dcac5 in init () from /lib64/libc.so.6 No symbol table info available. #11 0x00007fc9a6c1bbb0 in pthread_once () from /lib64/libpthread.so.0 No symbol table info available. #12 0x00007fc9a93dcbdc in backtrace () from /lib64/libc.so.6 No symbol table info available. #13 0x00007fc9a9347344 in __libc_message () from /lib64/libc.so.6 No symbol table info available. #14 0x00007fc9a934f053 in _int_free () from /lib64/libc.so.6 No symbol table info available. #15 0x00007fc9a933cff5 in fclose@@GLIBC_2.2.5 () from /lib64/libc.so.6 No symbol table info available. #16 0x00007fc9a88c29bf in _nss_files_gethostbyname4_r () from /lib64/libnss_files.so.2 No symbol table info available. #17 0x00007fc9a93aebb8 in gaih_inet () from /lib64/libc.so.6 No symbol table info available. #18 0x00007fc9a93b227d in getaddrinfo () from /lib64/libc.so.6 No symbol table info available. #19 0x00007fc99f7c86e4 in ?? () from /usr/lib64/libcurl.so.4 No symbol table info available. #20 0x00007fc99f7d2efa in ?? () from /usr/lib64/libcurl.so.4 No symbol table info available. #21 0x00007fc99f7d099b in ?? () from /usr/lib64/libcurl.so.4 No symbol table info available. #22 0x00007fc9a6c16dc5 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #23 0x00007fc9a93c8c9d in clone () from /lib64/libc.so.6 No symbol table info available.
(gdb) info locals No symbol table info available.
(gdb) list 1834 int proto; 1835 char *options; 1836 int ret; 1837 unsigned int seed; 1838 int rfd; 1839 int debug_save, debug_flag; 1840 int dont_fork_cnt; 1841 struct name_lst* n_lst; 1842 char *p; 1843 struct stat st = {0}; (gdb) ```
#### Log Messages
``` Aug 7 22:43:30 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [receive.c:173]: receive_msg(): core parsing of SIP message failed (67.191.157.42:5060/1) Aug 7 22:43:31 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event as-feature-event Aug 7 22:43:32 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:32 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:32 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event missed-call-summary Aug 7 22:43:36 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:36 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:37 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event as-feature-event Aug 7 22:43:38 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event as-feature-event Aug 7 22:43:40 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:40 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:44 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:44 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:47 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: ERROR: <core> [receive.c:173]: receive_msg(): core parsing of SIP message failed (67.191.157.42:5060/1) Aug 7 22:43:48 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:48 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:50 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event call-info Aug 7 22:43:52 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event as-feature-event Aug 7 22:43:52 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:52 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:52 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event as-feature-event Aug 7 22:43:52 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: NOTICE: presence [subscribe.c:1291]: handle_subscribe(): Unsupported presence event missed-call-summary Aug 7 22:43:56 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:43:56 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:43:57 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [receive.c:173]: receive_msg(): core parsing of SIP message failed (67.191.157.42:5060/1) Aug 7 22:44:00 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:44:00 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:44:00 ip-172-31-46-236 /usr/local/sbin/kamailio[9450]: ERROR: <core> [receive.c:173]: receive_msg(): core parsing of SIP message failed (67.191.157.42:5060/1) Aug 7 22:44:09 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:44:09 ip-172-31-46-236 /usr/local/sbin/kamailio[9449]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:44:10 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:44:10 ip-172-31-46-236 /usr/local/sbin/kamailio[9452]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:44:11 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: ERROR: maxfwd [mf_funcs.c:62]: is_maxfwd_present(): parsing MAX_FORWARD header failed! Aug 7 22:44:11 ip-172-31-46-236 /usr/local/sbin/kamailio[9451]: ERROR: <core> [msg_translator.c:2330]: build_res_buf_from_sip_req(): alas, parse_headers failed Aug 7 22:44:11 ip-172-31-46-236 /usr/local/sbin/kamailio[9477]: CRITICAL: <core> [pass_fd.c:277]: receive_fd(): EOF on 30 ```
#### SIP Traffic
N/A
### Possible Solutions
N/A
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
``` version: kamailio 4.4.6 (x86_64/linux) 4d49b3 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 4d49b3 compiled on 11:39:53 Jul 29 2017 with gcc 4.8.3 ```
* **Operating System**:
<!-- Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...; Kernel details (output of `uname -a`) -->
``` Linux ip-172-31-46-236 4.4.41-36.55.amzn1.x86_64 #1 SMP Wed Jan 18 01:03:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
It's Amazons Linux Distribution. I believe it's CentOS based. ```