[kamailio/kamailio] Core generated by Kamailio when a bad char is found in header (#1687)
24 Oct
2018
24 Oct
'18
6:40 p.m.
### Description
Kamailio dies complaining about a bad char in some header. I can say that I just updated
from the nightly-devel apt repo, this problem never happened in latest v5.1 release.
### Troubleshooting
#### Reproduction
I don't know how to reproduce yet.
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
[BT-FULL_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/kama…
[INFO-LOCALS_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/…
[LIST_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/kamaili…
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them
next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
The relevant log lines are:
```
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]:
parse_addr_spec(): unexpected char [<] in status 6: ["910609864"
<sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]:
get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]:
parse_headers(): bad header field [To: "910609864"
<sip:;:929<>64@138.99.136.3>;tag=as664d068c#015#012Call-ID:
032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]:
parse_addr_spec(): unexpected char [<] in status 6: ["910609864"
<sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]:
get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]:
parse_headers(): bad header field [To: "910609864"
<sip:;:929<>64@138.99.136.3>;tag=as664d068c#015#012Call-ID:
032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]:
parse_addr_spec(): unexpected char [<] in status 6: ["910609864"
<sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]:
get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]:
parse_headers(): bad header field [To: "910609864"
<sip:;:929<>64@138.99.136.3>;tag=as664d068c#015#012Call-ID:
032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: pv [pv_core.c:1892]: pv_get_hdr(): error parsing
headers
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]:
parse_addr_spec(): unexpected char [<] in status 6: ["910609864"
<sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]:
get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]:
parse_headers(): bad header field [To: "910609864"
<sip:;:929<>64@138.99.136.3>;tag=as664d068c#015#012Call-ID:
032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: pv [pv_core.c:704]: pv_get_callid(): cannot parse
Call-Id header
```
#### SIP Traffic
<!--
If the issue is exposed by processing specific SIP messages, grab them with ngrep or save
in a pcap file, then add them next, or attach to issue, or provide a link to download them
(e.g., to a pastebin site).
-->
I have enabled a mirror server to capture all traffic, once the problem happens again I
will be able to update this issue with SIP traffic.
### Possible Solutions
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
root@sbc01:/var/tmp# kamailio -v
version: kamailio 5.2.0-pre1 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535,
DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 6.3.0
root@sbc01:/var/tmp#
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04,
CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->
```
root@sbc01:/var/tmp# uname -a
Linux sbc01.vozelia.com.pa 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08)
x86_64 GNU/Linux
root@sbc01:/var/tmp#
root@sbc01:/var/tmp# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.5 (stretch)
Release: 9.5
Codename: stretch
root@sbc01:/var/tmp#
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687
24 Oct
24 Oct
6:45 p.m.
To add a more specific version as `5.2.0-pre1` doesn't say much...:
```
root@sbc01:/var/tmp# dpkg -l | grep kam
ii kamailio 5.2.0~pre1+0~20181020005754.1238+stretch amd64
very fast, dynamic and configurable SIP server
ii kamailio-dbg:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
very fast and configurable SIP server [debug symbols]
ii kamailio-extra-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
Extra modules for the Kamailio SIP Server
ii kamailio-geoip2-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
The geoip2 module for the Kamailio SIP Server
ii kamailio-mysql-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
MySQL database connectivity module for Kamailio
ii kamailio-outbound-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
SIP Outbound module for the Kamailio SIP server
ii kamailio-phonenum-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
phonenum modules for the Kamailio SIP server
ii kamailio-utils-modules:amd64 5.2.0~pre1+0~20181020005754.1238+stretch amd64
Utility functions for the Kamailio SIP server
root@sbc01:/var/tmp#
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-432853857
25 Oct
25 Oct
1:12 a.m.
The first two backtraces led to a function related to a new feature added in 5.2,
respectively statistics for replies per method. The issue was accessing CSeq header fields
without any checks if the header was properly parsed. I pushed a commit for it, referenced
above. You can try with latest master and see if now all ok.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-432916960
12:45 p.m.
Hi @miconda, I've just deployed K with this patch, I'll let you know if this
happens again.
Out of curiosity, if I wanted to test with the nightly builds, how can I know if for
example:
`5.2.0~pre1+0~20181025005757.1239+stretch` deb package contains commit 5e76302?
Thanks!
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433123753
3:49 p.m.
One idea: execute kamailio -V, this should output the build revision "id:". Then
you can see e.g. with git log if this revision already contains your wanted commit.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433182072
5:04 p.m.
Yeah this is my bad, it was there all the time, and only because you said so now I
actually see it:
```
root@sbc02:~# kamailio -V
version: kamailio 5.2.0-pre1 (x86_64/linux) 2ecf60-dirty
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER,
USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535,
DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 2ecf60 -dirty
compiled with gcc 6.3.0
root@sbc02:~#
```
Commit --> 2ecf60
Thanks @henningw
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433205047
5:04 p.m.
Closed #1687.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#event-1927363924
9:34 p.m.
Isn't "sip:;:929<>64@138.99.136.3" in the To header what's making
it fail?
On Thu, Oct 25, 2018, 22:05 Joel Serrano <notifications(a)github.com> wrote:
Closed #1687
<https://github.com/kamailio/kamailio/issues/1687>.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/kamailio/kamailio/issues/1687#event-1927363924>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AF36ZTx5k_vZi4fa0c4qEQAhUmCj0hawks5uoidagaJpZM4X5L1n>
.
_______________________________________________
Kamailio (SER) - Development Mailing List
sr-dev(a)lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev
26 Oct
26 Oct
12:59 p.m.
From the logs I also thought it was something in the
To: header but I don't know how to read the backtraces :(
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433475689
1 p.m.
Reopening for now!
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433475803
1 p.m.
Reopened #1687.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#event-1929351829
1:19 p.m.
@joelsdc - did you get any new crash after the commit I did? Even the error message is
related to To header, the issue was not checking if the CSeq is parsed. The core tries to
parse the mandatory headers like To, Call-ID, CSeq, ... if one is broken, parsing stops
there.
Unless you got new crash with the commit, there is no reason to open this issue. In
general, a log message may or may not be related to a crash, the gdb backtrace shows where
the crash happens. Otherwise you would apply a logic that if memory manager prints
"no more memory", then it is a bug inside it, not in the code using it.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#issuecomment-433481087
1:40 p.m.
Closed #1687.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687#event-1929432327
2123
days inactive
2125
days old
12 comments
4 participants
participants (4)
-
Daniel-Constantin Mierla -
David Villasmil -
Henning Westerholt
-
Joel Serrano