11 Sep
2015
11 Sep
'15
4:48 a.m.
caught it again
#### SYSLOG
```
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[tcp_read.c:1523]: handle_io(): received n=8 con=0x7f923c914778, fd=12
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[tcp_read.c:1332]: tcp_read_req(): content-length=240
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:616]: parse_msg(): SIP Reply (status):
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:618]: parse_msg(): version: <SIP/2.0>
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:620]: parse_msg(): status: <183>
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:622]: parse_msg(): reason: <Session Progress>
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 232, <branch> =
<z9hG4bKe733.06537d1bec1bc953b7b8ff05607a8672.1>; state=6
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 236, <i> =
<b61>; state=6
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 234, <received> =
<211.78.19.1>; state=6
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 235, <rport> =
<18490>; state=16
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:2642]: parse_via(): end of header reached, state=5
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:496]: parse_headers(): parse_headers: Via found, flags=2
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:498]: parse_headers(): parse_headers: this is the first via
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[receive.c:134]: receive_msg(): After parse_msg...
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 234, <received> =
<118.163.134.250>; state=6
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 232, <branch> =
<z9hG4bK.wMHXVdtOc>; state=6
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:1254]: parse_via_param(): Found param type 235, <rport> =
<43859>; state=16
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_via.c:2642]: parse_via(): end of header reached, state=5
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:496]: parse_headers(): parse_headers: Via found, flags=4
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:509]: parse_headers(): parse_headers: this is the second via
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: tm
[t_lookup.c:1011]: t_check_msg(): DEBUG: t_check_msg: msg id=114 global id=113 T
start=(nil)
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_addr_spec.c:172]: parse_to_param(): DEBUG: add_param: tag=as74c983b5
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/parse_addr_spec.c:894]: parse_addr_spec(): end of header reached, state=29
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:173]: get_hdr_field(): DEBUG: get_hdr_field: <To> [55];
uri=[sip:5777015@211.78.19.1]
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:175]: get_hdr_field(): DEBUG: to body ["0979708746"
<sip:5777015@211.78.19.1>]
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: <core>
[parser/msg_parser.c:153]: get_hdr_field(): get_hdr_field: cseq <CSeq>: <20>
<INVITE>
Sep 11 10:15:18 sip1 /home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: tm [t_lookup.c:888]:
t_reply_matching(): DEBUG: t_reply_matching: hash 13182 label 0 branch 1
Sep 11 10:15:18 sip1 kernel: [41763585.057940] kamailio[123969] general protection
ip:7f923ba4cd1b sp:7fff5bdaef50 error:0 in tm.so[7f923b9e4000+114000]
Sep 11 10:15:39 sip1 /home/pkg/kamailio/sbin/kamailio[123971]: CRITICAL: <core>
[pass_fd.c:275]: receive_fd(): EOF on 108
Sep 11 10:15:39 sip1 /home/pkg/kamailio/sbin/kamailio[123971]: DEBUG: <core>
[tcp_main.c:3448]: handle_ser_child(): dead child 72, pid 123969 (shutting down?)
Sep 11 10:15:39 sip1 /home/pkg/kamailio/sbin/kamailio[123971]: DEBUG: <core>
[io_wait.h:598]: io_watch_del(): DBG: io_watch_del (0x9e9d40, 108, -1, 0x0) fd_no=106
called
Sep 11 10:15:39 sip1 /home/pkg/kamailio/sbin/kamailio[123896]: ALERT: <core>
[main.c:728]: handle_sigs(): child process 123969 exited by a signal 11
```
#### BACKTRACE core.kamailio.sig11.123969.1441937718
```
#0 0x00007f923ba4cd1b in t_reply_matching (p_msg=0x7f937f050140, p_branch=0x7fff5bdaf340)
at t_lookup.c:901
p_cell = 0x1e43e000000af
hash_index = 13182
entry_label = 0
branch_id = 1
hashi = 0x172ed5e
"e733.06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP
192.168.2.84:38483;received=118.163.134.250;branch=z9hG4bK.wMHXVdtOc;rport=43859\r\nRecord-Route:
<sip:21"...
branchi = 0x172ed84 "1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP
192.168.2.84:38483;received=118.163.134.250;branch=z9hG4bK.wMHXVdtOc;rport=43859\r\nRecord-Route:
<sip:211.78.19.1:5003;r2=on;lr=on;ftag=0KeO-~"...
p = 0x172ed84 "1;i=b61;received=211.78.19.1;rport=18490\r\nVia: SIP/2.0/TCP
192.168.2.84:38483;received=118.163.134.250;branch=z9hG4bK.wMHXVdtOc;rport=43859\r\nRecord-Route:
<sip:211.78.19.1:5003;r2=on;lr=on;ftag=0KeO-~"...
n = 0x172ed85 ";i=b61;received=211.78.19.1;rport=18490\r\nVia: SIP/2.0/TCP
192.168.2.84:38483;received=118.163.134.250;branch=z9hG4bK.wMHXVdtOc;rport=43859\r\nRecord-Route:
<sip:211.78.19.1:5003;r2=on;lr=on;ftag=0KeO-~i"...
hash_bucket = 0x7f923c423088
hashl = 4
branchl = 1
scan_space = 1
cseq_method = {
s = 0x172ef60 "INVITE\r\nServer: PBX\r\nAllow: INVITE, ACK, CANCEL,
OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\nSupported: replaces,
timer\r\nContact:
<sip:886979708746@211.78.19.2:5070;transport=TCP>\r\nContent-"..., len = 6}
req_method = {s = 0x4188b0
"1\355I\211\321^H\211\342H\203\344\360PTI\307\300@\202n", len = 1541078944}
loopi = 0x172ed63
"06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP
192.168.2.84:38483;received=118.163.134.250;branch=z9hG4bK.wMHXVdtOc;rport=43859\r\nRecord-Route:
<sip:211.78."...
loopl = 32
is_cancel = 0
__FUNCTION__ = "t_reply_matching"
#1 0x00007f923ba4fb0f in t_check_msg (p_msg=0x7f937f050140, param_branch=0x7fff5bdaf340)
at t_lookup.c:1069
local_branch = 1541075456
canceled = -1
ret = 0
__FUNCTION__ = "t_check_msg"
#2 0x00007f923ba50489 in t_check (p_msg=0x7f937f050140, param_branch=0x7fff5bdaf340) at
t_lookup.c:1111
ret = 0
#3 0x00007f923ba9794b in reply_received (p_msg=0x7f937f050140) at t_reply.c:2118
msg_status = 0
last_uac_status = 23627184
ack = 0x7fff5bdaf410 "\020\365\332[\377\177"
ack_len = 32767
branch = 3
reply_status = 6440375
onreply_route = 952550563
cancel_data = {cancel_bitmap = 4, reason = {cause = 320, u = {text = {s = 0xb
<Address 0xb out of bounds>, len = -88}, e2e_cancel = 0xb, packed_hdrs = {s = 0xb
<Address 0xb out of bounds>,
len = -88}}}}
uac = 0x7f937f050160
t = 0x762130
lack_dst = {send_sock = 0x7fff5bdaf410, to = {s = {sa_family = 24128, sa_data =
"\212\276\223\177\000\000\000\000\000\000\000\000\000"}, sin = {sin_family =
24128, sin_port = 48778, sin_addr = {
s_addr = 32659}, sin_zero = "\000\000\000\000\000\000\000"},
sin6 = {sin6_family = 24128, sin6_port = 48778, sin6_flowinfo = 32659, sin6_addr =
{__in6_u = {
__u6_addr8 =
"\000\000\000\000\000\000\000\000\377\377\377\377\000\000\000", __u6_addr16 =
{0, 0, 0, 0, 65535, 65535, 0, 0}, __u6_addr32 = {0, 0, 4294967295, 0}}}, sin6_scope_id =
11}},
id = 0, proto = 64 '@', send_flags = {f = 94 '^', blst_imask =
138 '\212'}}
backup_user_from = 0x172f146
backup_user_to = 0x7f937e9435c8
backup_domain_from = 0x7f93be5f6a6d
backup_domain_to = 0x172ee11
backup_uri_from = 0x7f937f050140
backup_uri_to = 0x0
backup_xavps = 0x7f9300000003
replies_locked = 14385
branch_ret = 0
prev_branch = 32659
blst_503_timeout = 2131034432
hf = 0x763058
onsend_params = {req = 0x4004188b0, rpl = 0x75fc90, param = 0x0, code = 0, flags =
3, branch = 0, t_rbuf = 0x7fff5bdaf390, dst = 0x631592, send_buf = {
s = 0x1745810 "<31>Sep 11 10:15:18
/home/pkg/kamailio/sbin/kamailio[123969]: DEBUG: tm [t_lookup.c:888]: t_reply_matching():
DEBUG: t_reply_matching: hash 13182 label 0 branch 1\n",
len = 24401328}}
ctx = {rec_lev = 1541075856, run_flags = 32767, last_retcode = 4294832, jmp_env =
{{__jmpbuf = {140734734466976, 0, 0, 140272530845376, 206158430248, 140734734463712,
140734734463520, 0},
__mask_was_saved = 1541075664, __saved_mask = {__val = {6422143, 7315080,
7739304, 7745634, 0, 140734734464128, 11, 18446744073709551528, 24401920, 8208,
1400183748624, 140271467956440,
7733692, 64, 24309265, 1441937718}}}}}
__FUNCTION__ = "reply_received"
#4 0x00000000004901cc in do_forward_reply (msg=0x7f937f050140, mode=0) at forward.c:747
new_buf = 0x0
dst = {send_sock = 0x0, to = {s = {sa_family = 0, sa_data = '\000'
<repeats 13 times>}, sin = {sin_family = 0, sin_port = 0, sin_addr = {s_addr = 0},
sin_zero = "\000\000\000\000\000\000\000"},
sin6 = {sin6_family = 0, sin6_port = 0, sin6_flowinfo = 0, sin6_addr =
{__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0,
0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0,
0}}}, sin6_scope_id = 0}}, id = 0, proto = 0 '\000',
send_flags = {f = 0 '\000', blst_imask = 0 '\000'}}
new_len = 0
r = 2
ip = {af = 40, len = 48, u = {addrl = {140734734464320, 140734734464128}, addr32 =
{1541076288, 32767, 1541076096, 32767}, addr16 = {62784, 23514, 32767, 0, 62592, 23514,
32767, 0},
addr = "@\365\332[\377\177\000\000\200\364\332[\377\177\000"}}
s = 0x7f9300000003 ""
len = 32658
__FUNCTION__ = "do_forward_reply"
#5 0x0000000000491856 in forward_reply (msg=0x7f937f050140) at forward.c:849
No locals.
#6 0x000000000050e062 in receive_msg (
buf=0x172ed10 "SIP/2.0 183 Session Progress\r\nVia: SIP/2.0/TCP
211.78.19.1:5003;branch=z9hG4bKe733.06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP 192.168.2.84:38483;receiv"..., len=1078, rcv_info=0x7f923c914790) at
receive.c:255
msg = 0x7f937f050140
ctx = {rec_lev = 0, run_flags = 32658, last_retcode = 28800, jmp_env = {{__jmpbuf
= {23627184, 11, -88, 4629999147008, 8208, 0, 4294832, 140734734466976}, __mask_was_saved
= 1541076576,
__saved_mask = {__val = {6234849, 24401936, 0, 24401936, 130, 24401936,
1441937718, 8208, 140266058106744, 140734734465168, 4294832, 140734734466976, 0, 0,
140272530845376, 206158430256}}}}}
ret = -88
inb = {
s = 0x172ed10 "SIP/2.0 183 Session Progress\r\nVia: SIP/2.0/TCP
211.78.19.1:5003;branch=z9hG4bKe733.06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP 192.168.2.84:38483;receiv"..., len = 1078}
__FUNCTION__ = "receive_msg"
#7 0x00000000005f65fc in receive_tcp_msg (
tcpbuf=0x7f923c914a68 "SIP/2.0 183 Session Progress\r\nVia: SIP/2.0/TCP
211.78.19.1:5003;branch=z9hG4bKe733.06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP 192.168.2.84:38483;receiv"..., len=1078, rcv_info=0x7f923c914790,
con=0x7f923c914778) at tcp_read.c:1247
buf = 0x172ed10 "SIP/2.0 183 Session Progress\r\nVia: SIP/2.0/TCP
211.78.19.1:5003;branch=z9hG4bKe733.06537d1bec1bc953b7b8ff05607a8672.1;i=b61;received=211.78.19.1;rport=18490\r\nVia:
SIP/2.0/TCP 192.168.2.84:38483;receiv"...
bsize = 65535
blen = 65535
__FUNCTION__ = "receive_tcp_msg"
#8 0x00000000005f7d9b in tcp_read_req (con=0x7f923c914778, bytes_read=0x7fff5bdaf96c,
read_flags=0x7fff5bdaf968) at tcp_read.c:1401
bytes = 1078
total_bytes = 1078
resp = 1
size = 140734734465024
req = 0x7f923c9147f8
dst = {send_sock = 0x14, to = {s = {sa_family = 1, sa_data =
"\000\000\001\000\000\000\210\236o\000\000\000\000"}, sin = {sin_family = 1,
sin_port = 0, sin_addr = {s_addr = 1},
sin_zero = "\210\236o\000\000\000\000"}, sin6 = {sin6_family = 1,
sin6_port = 0, sin6_flowinfo = 1, sin6_addr = {__in6_u = {
__u6_addr8 =
"\210\236o\000\000\000\000\000\025\032u\000\000\000\000", __u6_addr16 = {40584,
111, 0, 0, 6677, 117, 0, 0}, __u6_addr32 = {7315080, 0, 7674389, 0}}},
sin6_scope_id = 7678414}}, id = 0, proto = 8 '\b', send_flags = {f =
0 '\000', blst_imask = 0 '\000'}}
c = 0 '\000'
ret = 1541076976
__FUNCTION__ = "tcp_read_req"
#9 0x00000000005fa3e5 in handle_io (fm=0x7f937e96f900, events=1, idx=-1) at
tcp_read.c:1568
ret = 8
n = 8
read_flags = 1
con = 0x7f923c914778
s = 12
resp = 140266057669928
t = 1015717160
__FUNCTION__ = "handle_io"
#10 0x00000000005ee8c0 in io_wait_loop_epoll (h=0xa2dfe0, t=2, repeat=1) at
io_wait.h:1061
n = 1
r = 0
fm = 0x7f937e96f900
revents = 1
__FUNCTION__ = "io_wait_loop_epoll"
#11 0x00000000005fbe1a in tcp_receive_loop (unix_sock=112) at tcp_read.c:1739
__FUNCTION__ = "tcp_receive_loop"
#12 0x00000000005e54e1 in tcp_init_children () at tcp_main.c:4787
r = 22
i = 23
reader_fd_1 = 112
pid = 0
si_desc = "tcp receiver
(generic)\000\000ygY\000\000\000\000\000\000\374\332[\377\177\000\000R\342B\000\000\000\000\000\340\373\332[\377\177\000\000\260\343R\276\223\177\000\000
2\224~\223\177\000\000\210W\364/\000\000\000\000\217sp\000\000\000\000\000\020\061U~\223\177\000\000)\000\000\000*\000\000\000Uz;\023\344\001\000\037\344\001\000aިf\210W\364/\000\000\000"
si = 0x0
__FUNCTION__ = "tcp_init_children"
#13 0x00000000004ab258 in main_loop () at main.c:1658
i = 24
pid = 123923
si = 0x0
si_desc = "udp receiver child=23
sock=211.78.19.1:5003\000\000\000\000\000\240\377\332[\377\177", '\000'
<repeats 18 times>"\300,
n_\276\223\177\000\000\060\000\000\000\060\000\000\000@\037\224~\223\177\000\000\320\374\332[\377\177\000\000\260\004\066<\222\177\000\000\210W\364/\000\000\000\000\260\210A\000\001\000\000"
nrprocs = 24
__FUNCTION__ = "main_loop"
#14 0x00000000004b0b5b in main (argc=13, argv=0x7fff5bdaffa8) at main.c:2533
cfg_stream = 0x1682010
c = -1
r = 0
tmp = 0x7fff5bdb07c3 ""
tmp_len = 32659
port = -1091676304
proto = 0
options = 0x707db8
":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
ret = -1
seed = 2623255322
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 0
n_lst = 0x76
p = 0x7fff5bdafeae ""
__FUNCTION__ = "main"
```
---
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/322#issuecomment-139434734