Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
Did you get a core dump, can you send the backtrace?
Daniel
On 10/9/12 11:00 AM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
I can not locate where the core is (wasn'i it in /tmp?) but I see the following statement in the log:
Oct 9 12:44:21 kamailio1 kamailio[17420]: ALERT: <core> [main.c:788]: core was generated
so i guess somewhere in the disk there is a core file....
Thanks a lot, Daniel, Samuel.
On 9 October 2012 12:08, Daniel-Constantin Mierla miconda@gmail.com wrote:
Did you get a core dump, can you send the backtrace?
Daniel
On 10/9/12 11:00 AM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
If you provided -w parameter, then it is at that path. If not, it should be in '/' or in the OS-configured corefiles directory.
Daniel
On 10/9/12 12:45 PM, samuel wrote:
I can not locate where the core is (wasn'i it in /tmp?) but I see the following statement in the log:
Oct 9 12:44:21 kamailio1 kamailio[17420]: ALERT: <core> [main.c:788]: core was generated
so i guess somewhere in the disk there is a core file....
Thanks a lot, Daniel, Samuel.
On 9 October 2012 12:08, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
Did you get a core dump, can you send the backtrace? Daniel On 10/9/12 11:00 AM, samuel wrote:
Hi folks, I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user. The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1) basically for all incoming messages, a route is called with the following actions: route[HOMER] { setflag(22); sip_trace(); } I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error: <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28 There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it. Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash... Thanks in advance! Samuel. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -http://www.asipto.com http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> -http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 -http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 -http://asipto.com/u/katu
This is the output of the core file:
(gdb) bt full #0 do_action (h=<value optimized out>, a=0x485f3832315f4d43, msg=<value optimized out>) at action.c:340 ret = <value optimized out> v = <value optimized out> dst = {send_sock = 0x1bca1e0, to = {s = {sa_family = 0, sa_data = "\000\000\003\000\000\000\000\000\000\000\000\000\000"}, sin = { sin_family = 0, sin_port = 0, sin_addr = {s_addr = 3}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 0, sin6_port = 0, sin6_flowinfo = 3, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, id = 0, proto = 0 '\000', send_flags = {f = 0 '\000', blst_imask = 0 '\000'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0x7f5392d0de40 "", len = 4}, passwd = {s = 0x0, len = 0}, host = {s = 0x7f5392617aa0 "\230\216^\222S\177", len = -1834849744}, port = {s = 0x4 <Address 0x4 out of bounds>, len = -1831805376}, params = { s = 0x7fffc8687450 "\250\v\213", len = 0}, sip_params = {s = 0x0, len = 0}, headers = {s = 0x1bc6510 "\004", len = -1834849972}, port_no = 0, proto = 0, type = ERROR_URI_T, flags = 2460386843, transport = { s = 0x7f538994e050 "%u.%u.%u.%u.in-addr.arpa", len = 0}, ttl = {s = 0x7fffc8687558 "CM_128_H\373\377\377\377", len = 3}, user_param = {s = 0x7fffc8687500 "", len = 0}, maddr = {s = 0x10 <Address 0x10 out of bounds>, len = 29139424}, method = { s = 0x7f53925e8e98 "\340\241\274\001", len = -1845602791}, lr = {s = 0x1bcac60 "", len = -1827450870}, r2 = { s = 0x3 <Address 0x3 out of bounds>, len = 9112488}, gr = {s = 0x100000002 <Address 0x100000002 out of bounds>, len = -932678384}, transport_val = {s = 0x0, len = 0}, ttl_val = {s = 0x1d3764dcf <Address 0x1d3764dcf out of bounds>, len = -932678360}, user_param_val = {s = 0xd827590 <Address 0xd827590 out of bounds>, len = 7}, maddr_val = { s = 0x7fffc8687540 "\bzh\310\377\177", len = -932678316}, method_val = {s = 0x7f53906236a0 "\024", len = 0}, lr_val = { s = 0x8b0ba8 "100 Giving a try\r\nVia: SIP/2.0/UDP 172.30.1.76;branch=z9hG4bK3568.03a3e98.0;i=1\r\nVia: SIP/2.0/tls 172.30.1.241:54666;rport=54666;branch=z9hG4bKPj0n0hVs50Gyl6TfXJPcSI4xeJnWZ3QUKc\r\nFrom: "Samuel" <sip:s"..., len = 29117296}, r2_val = {s = 0x0, len = 0}, gr_val = {s = 0x0, len = 0}} next_hop = {user = {s = 0x2001400000040 <Address 0x2001400000040 out of bounds>, len = 1349779723}, passwd = { s = 0x2fd80400a <Address 0x2fd80400a out of bounds>, len = 65556}, host = { s = 0xffac000000000000 <Address 0xffac000000000000 out of bounds>, len = 151002878}, port = { s = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, len = 1110}, params = { s = 0x100007fffff0000 <Address 0x100007fffff0000 out of bounds>, len = 0}, sip_params = {s = 0x7fffc8687607 "", len = -1834348756}, headers = {s = 0x2000300000014 <Address 0x2000300000014 out of bounds>, len = 1349779723}, port_no = 0, proto = 0, type = SIP_URI_T, flags = 65556, transport = {s = 0x0, len = 16777216}, ttl = { s = 0xffffffffffffffff <Address 0xffffffffffffffff out of bounds>, len = -1834588776}, user_param = { s = 0x2001400000040 <Address 0x2001400000040 out of bounds>, len = 0}, maddr = { s = 0x2fd80400a <Address 0x2fd80400a out of bounds>, len = 65556}, method = {s = 0x0, len = -932678912}, lr = { s = 0xffffffff <Address 0xffffffff out of bounds>, len = -932691472}, r2 = {s = 0x7fffc86841e8 "\020e\274\001", len = -1}, gr = {s = 0x7fffc868733c "S\177", len = 29123664}, transport_val = {s = 0x7fffc8685210 "", len = -932691488}, ttl_val = { s = 0x7fff00000001 <Address 0x7fff00000001 out of bounds>, len = -932678880}, user_param_val = { s = 0x10 <Address 0x10 out of bounds>, len = 32}, maddr_val = {s = 0x7f5392d0de98 "\260\v\276\001", len = -1831805272}, method_val = {s = 0x7f539205204f "latin1_swedish_ci", len = 4}, lr_val = {s = 0x2c8687707 <Address 0x2c8687707 out of bounds>, len = -1831805368}, r2_val = {s = 0x40 <Address 0x40 out of bounds>, len = 0}, gr_val = {s = 0x0, len = -1834602969}} u = <value optimized out> port = <value optimized out> i = <value optimized out> flags = <value optimized out> avp = <value optimized out> st = {flags = 0, id = 0, name = {n = 0, s = {s = 0x0, len = 0}, re = 0x0}, avp = 0x0} sct = <value optimized out> ---Type <return> to continue, or q <return> to quit--- sjt = <value optimized out> rve = <value optimized out> mct = <value optimized out> rv = <value optimized out> rv1 = <value optimized out> c1 = {cache_type = RV_CACHE_SELECT, val_type = RV_ACTION_ST, c = {avp_val = {n = 3, s = { s = 0x300000003 <Address 0x300000003 out of bounds>, len = 3}, re = 0x300000003}, pval = {rs = { s = 0x300000003 <Address 0x300000003 out of bounds>, len = 3}, ri = 3, flags = 6}}, i2s = "\000\000\000\000\000\000\000\000\210\242\274\001\000\000\000\000\001\000\000\001\000"} s = {s = 0x0, len = 0} srevp = {0x2, 0x0} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #1 0x000000000041e406 in run_actions (h=0x7fffc8688230, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x485f3832315f4d43 ret = -5 mod = <value optimized out> ms = 4294967291 #2 0x000000000041742c in do_action (h=<value optimized out>, a=0x7f5392617b50, msg=<value optimized out>) at action.c:761 ret = -5 v = <value optimized out> dst = {send_sock = 0x6, to = {s = {sa_family = 2, sa_data = "\000\000\000\000\000\000P|h\310\377\177\000"}, sin = {sin_family = 2, sin_port = 0, sin_addr = {s_addr = 0}, sin_zero = "P|h\310\377\177\000"}, sin6 = {sin6_family = 2, sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = "P|h\310\377\177\000\000\240\203a\222S\177\000", __u6_addr16 = { 31824, 51304, 32767, 0, 33696, 37473, 32595, 0}, __u6_addr32 = {3362290768, 32767, 2455864224, 32595}}}, sin6_scope_id = 2455824448}}, id = 32595, proto = 48 '0', send_flags = {f = 130 '\202', blst_imask = 104 'h'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0x7fffc8687e50 "", len = -932676032}, passwd = {s = 0x5a65e3 "", len = -1835130845}, host = { s = 0x7f538a8731c4 "\002", len = -1839133152}, port = {s = 0x1 <Address 0x1 out of bounds>, len = 4318352}, params = { s = 0x7f5391b710b5 "%.*s=", len = -932676100}, sip_params = {s = 0x0, len = -1825307208}, headers = {s = 0x7f5393340000 "", len = 4254337}, port_no = 1656, proto = 37532, type = 32595, flags = 4219224, transport = { s = 0x100000000 <Address 0x100000000 out of bounds>, len = 1684}, ttl = {s = 0x7fffc8687df0 "", len = -1825262464}, user_param = {s = 0x7f53906236a0 "\024", len = -932676032}, maddr = {s = 0x7f5392610e20 "\251\003", len = -1825263320}, method = {s = 0x1b81fa72 <Address 0x1b81fa72 out of bounds>, len = -1839132912}, lr = {s = 0x7f539260fcb0 "\001", len = -932676032}, r2 = {s = 0x5a65e3 "", len = 5863229}, gr = {s = 0x8092e0 "1\350Y", len = 4290462}, transport_val = { s = 0x7fffc8687f08 "172.30.1.87:5060", len = -932675864}, ttl_val = { s = 0x7f5300000000 <Address 0x7f5300000000 out of bounds>, len = -932675892}, user_param_val = {s = 0x0, len = -1825262464}, maddr_val = {s = 0x7fffc86879c0 "hl\214", len = -932675704}, method_val = {s = 0x7fffc8688230 "\003", len = -932675024}, lr_val = {s = 0x7fffc8687f88 "\001", len = 5863229}, r2_val = {s = 0x8092e0 "1\350Y", len = 4931420}, gr_val = { ---Type <return> to continue, or q <return> to quit--- s = 0xbec27c38f7bcc18a <Address 0xbec27c38f7bcc18a out of bounds>, len = 1624100772}} next_hop = {user = {s = 0x0, len = 0}, passwd = {s = 0x7fff00000001 <Address 0x7fff00000001 out of bounds>, len = -1839105408}, host = {s = 0x7f539260e840 "/", len = -932675024}, port = {s = 0x7fffc8687f88 "\001", len = 5863229}, params = { s = 0x8092e0 "1\350Y", len = 4892947}, sip_params = {s = 0x7fffc8687773 "", len = -932676872}, headers = { s = 0x7f538e241fdb "%s: %s%.*stimestamp=%lu%s", len = 0}, port_no = 32768, proto = 51304, type = 32767, flags = 2449501538, transport = {s = 0x3a28d <Address 0x3a28d out of bounds>, len = -932676296}, ttl = {s = 0x7fffc8687e40 "", len = 4931461}, user_param = {s = 0x3a28d <Address 0x3a28d out of bounds>, len = -1845460557}, maddr = {s = 0x1 <Address 0x1 out of bounds>, len = -1839104096}, method = {s = 0x7f53929b94d8 "RD", len = -1825308672}, lr = { s = 0x7fff00000000 <Address 0x7fff00000000 out of bounds>, len = -1}, r2 = {s = 0xb <Address 0xb out of bounds>, len = 1}, gr = {s = 0x806ca0 "\360\037\236\222S\177", len = 0}, transport_val = {s = 0x59773d "corefunc", len = 8426208}, ttl_val = { s = 0x0, len = -1827432686}, user_param_val = {s = 0x7fff00000001 <Address 0x7fff00000001 out of bounds>, len = 0}, maddr_val = {s = 0xffffffff <Address 0xffffffff out of bounds>, len = -1835297576}, method_val = { s = 0x7fffc8687eb0 "\004\177h\310\377\177", len = -1839133152}, lr_val = {s = 0x1 <Address 0x1 out of bounds>, len = -932676032}, r2_val = {s = 0x5a65e3 "", len = -1827408347}, gr_val = {s = 0x1 <Address 0x1 out of bounds>, len = 1}} u = <value optimized out> port = <value optimized out> i = 9 flags = <value optimized out> avp = <value optimized out> st = {flags = 3362291592, id = 32767, name = {n = 4948316, s = { s = 0x4b815c "\205\300A\211\307\017\211u\376\377\377\350\004\323\370\377\205\300f\220\017\210\272\005", len = -1831821280}, re = 0x4b815c}, avp = 0x7fffc8687c2f} sct = <value optimized out> sjt = <value optimized out> rve = <value optimized out> mct = <value optimized out> rv = 0x7f538a96e460 rv1 = <value optimized out> c1 = {cache_type = 2179837245, val_type = 3476970084, c = {avp_val = {n = -1868273171, s = { s = 0x7f5390a469ed "\211C\f1\300[\303fff.\017\037\204", len = -1973731328}, re = 0x7f5390a469ed}, pval = {rs = { s = 0x7f5390a469ed "\211C\f1\300[\303fff.\017\037\204", len = -1973731328}, ri = -1834707850, flags = 32595}}, i2s = "hl\214\000\000\000\000\000h\000\000\000\000\000\000\000\b\r\214\000\000"} s = {s = 0x7fffc8687cd0 "Пa\222S\177", len = -932676544} srevp = {0x3000000018, 0x7fffc8687d90} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #3 0x000000000041e406 in run_actions (h=0x7fffc8688230, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7f5392617b50 ret = 0 mod = <value optimized out> ms = 0 #4 0x000000000041779e in do_action (h=<value optimized out>, a=0x7f5392618a70, msg=<value optimized out>) at action.c:1136 ret = 1 v = 1 ---Type <return> to continue, or q <return> to quit--- dst = {send_sock = 0x3a70647500000000, to = {s = {sa_family = 14129, sa_data = "2.30.1.87:5060"}, sin = {sin_family = 14129, sin_port = 11826, sin_addr = {s_addr = 825110579}, sin_zero = ".87:5060"}, sin6 = {sin6_family = 14129, sin6_port = 11826, sin6_flowinfo = 825110579, sin6_addr = {__in6_u = {__u6_addr8 = ".87:5060\000\000\000\000\000\000\000", __u6_addr16 = { 14382, 14903, 12341, 12342, 0, 0, 0, 0}, __u6_addr32 = {976697390, 808857653, 0, 0}}}, sin6_scope_id = 0}}, id = 0, proto = 0 '\000', send_flags = {f = 0 '\000', blst_imask = 0 '\000'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0x57011eac13c40002 <Address 0x57011eac13c40002 out of bounds>, len = 0}, passwd = {s = 0x0, len = 0}, host = { s = 0x50740211 <Address 0x50740211 out of bounds>, len = 492511}, port = { s = 0x783df50740211 <Address 0x783df50740211 out of bounds>, len = -1969815551}, params = {s = 0x0, len = -932676064}, sip_params = {s = 0x4c011eac57011eac <Address 0x4c011eac57011eac out of bounds>, len = 0}, headers = { s = 0xc413c41311021002 <Address 0xc413c41311021002 out of bounds>, len = -1839142848}, port_no = 32832, proto = 51304, type = 32767, flags = 2325169800, transport = {s = 0x7f538a959700 "\026", len = 72}, ttl = { s = 0x7fffc8687ed0 "udp:172.30.1.76:5060", len = -1839143408}, user_param = {s = 0x20 <Address 0x20 out of bounds>, len = -932676064}, maddr = {s = 0x0, len = -1982407143}, method = {s = 0x0, len = 0}, lr = {s = 0x0, len = 0}, r2 = {s = 0x0, len = 0}, gr = {s = 0x0, len = 0}, transport_val = {s = 0x0, len = 0}, ttl_val = {s = 0x0, len = 9112480}, user_param_val = { s = 0x1ad <Address 0x1ad out of bounds>, len = 9112789}, maddr_val = {s = 0x20 <Address 0x20 out of bounds>, len = -1969906352}, method_val = {s = 0x6 <Address 0x6 out of bounds>, len = -932675760}, lr_val = { s = 0x3 <Address 0x3 out of bounds>, len = -1982401395}, r2_val = { s = 0x8b0c84 "RtS65yC7G7LvJqZ-V3486e7HiRthsjgU\r\nTo: < sip:00555654646848@172.30.1.76>\r\nCall-ID: OsN.Pg6z4jgmqhrELm98-7AIPCVjxAhg\r\nCSeq: 12989 INVITE\r\nServer: OpenSIPS (1.8.0-tls (x86_64/linux))\r\nContent-Length: 0\r\n\r"..., len = 32}, gr_val = { s = 0x7fffc8687f04 "udp:172.30.1.87:5060", len = 20}} next_hop = {user = {s = 0x7f53925af010 "", len = 5424982}, passwd = {s = 0x1 <Address 0x1 out of bounds>, len = -1868273171}, host = {s = 0x7f53925af010 "", len = 5424982}, port = {s = 0x1 <Address 0x1 out of bounds>, len = -1839168248}, params = { s = 0x7f53926084f0 "", len = -1980290472}, sip_params = {s = 0x1 <Address 0x1 out of bounds>, len = -932676064}, headers = { s = 0x8c0d00 "\020\360Z\222S\177", len = -1980290472}, port_no = 11896, proto = 37472, type = 32595, flags = 2312528666, transport = {s = 0x43e <Address 0x43e out of bounds>, len = -1839104816}, ttl = {s = 0x8c0d00 "\020\360Z\222S\177", len = -932676464}, user_param = {s = 0x0, len = -1839104816}, maddr = {s = 0x0, len = -1980291912}, method = { s = 0x1be <Address 0x1be out of bounds>, len = 300}, lr = {s = 0x8c0d00 "\020\360Z\222S\177", len = -932676400}, r2 = { s = 0x0, len = -1868273171}, gr = {s = 0x7f53925af010 "", len = 5426506}, transport_val = { s = 0x8b0c7f ";tag=RtS65yC7G7LvJqZ-V3486e7HiRthsjgU\r\nTo: < sip:00555654646848@172.30.1.76>\r\nCall-ID: OsN.Pg6z4jgmqhrELm98-7AIPCVjxAhg\r\nCSeq: 12989 INVITE\r\nServer: OpenSIPS (1.8.0-tls (x86_64/linux))\r\nContent-Length:"..., len = -1839143408}, ttl_val = { s = 0x7f5392619fd0 "\b", len = 1680015362}, user_param_val = {s = 0x0, len = 0}, maddr_val = {s = 0x0, len = 1}, method_val = { s = 0x4c011eac13c50002 <Address 0x4c011eac13c50002 out of bounds>, len = 9112741}, lr_val = {s = 0x5a65e3 "", len = 0}, r2_val = {s = 0x4c011eac13c40002 <Address 0x4c011eac13c40002 out of bounds>, len = 0}, gr_val = {s = 0x0, len = 0}} u = <value optimized out> port = <value optimized out> i = <value optimized out> flags = <value optimized out> avp = <value optimized out> st = {flags = 0, id = 0, name = {n = 1349779985, s = {s = 0x50740211 <Address 0x50740211 out of bounds>, len = 492510}, re = 0x50740211}, avp = 0x48} sct = <value optimized out> sjt = <value optimized out> rve = 0x7f53926183a0 mct = <value optimized out> rv = <value optimized out> ---Type <return> to continue, or q <return> to quit--- rv1 = <value optimized out> c1 = {cache_type = 3362291408, val_type = 32767, c = {avp_val = {n = 20, s = {s = 0x14 <Address 0x14 out of bounds>, len = 980444277}, re = 0x14}, pval = {rs = {s = 0x14 <Address 0x14 out of bounds>, len = 980444277}, ri = 825110579, flags = 976631598}}, i2s = "5060", '\000' <repeats 17 times>} s = {s = 0x800000 <Address 0x800000 out of bounds>, len = -1969817840} srevp = {0x303031, 0x7f538a96eb08} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #5 0x000000000041e406 in run_actions (h=0x7fffc8688230, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7f5392618a70 ret = 1 mod = <value optimized out> ms = 1 #6 0x000000000041e9b0 in run_top_route (a=0x7f5392617870, msg=0x7f539260e840, c=<value optimized out>) at action.c:1729 ctx = {rec_lev = 34, run_flags = 0, last_retcode = 9112823, jmp_env = {{__jmpbuf = {9112909, 5440393, 781684047872, 140736555680776, 9112835, 0, 139996784222568, 139996914837568}, __mask_was_saved = -1872608172, __saved_mask = {__val = { 140736555680776, 429, 140736555681576, 28, 139996878928701, 139996784114152, 139996814656151, 139996784114155, 139996784035848, 139996928669352, 139996878939331, 139996914927352, 139996817766136, 4591320040374, 139996914837568, 139996914837568}}}}} p = 0x7fffc8688230 ret = <value optimized out> sfbk = 0 #7 0x00007f53903f20b2 in reply_received (p_msg=0x7f539260e840) at t_reply.c:2234 msg_status = 100 last_uac_status = 0 ack = <value optimized out> ack_len = 16 branch = 0 reply_status = <value optimized out> onreply_route = 0 cancel_data = {cancel_bitmap = 0, reason = {cause = 0, u = {text = {s = 0x0, len = 9112511}, e2e_cancel = 0x0, packed_hdrs = { s = 0x0, len = 9112511}}}} t = 0x7f538a974688 lack_dst = {send_sock = 0x1b0, to = {s = {sa_family = 18168, sa_data = "b\222S\177\000\000\200\204h\310\377\177\000"}, sin = { sin_family = 18168, sin_port = 37474, sin_addr = {s_addr = 32595}, sin_zero = "\200\204h\310\377\177\000"}, sin6 = { sin6_family = 18168, sin6_port = 37474, sin6_flowinfo = 32595, sin6_addr = {__in6_u = { __u6_addr8 = "\200\204h\310\377\177\000\000\370Fb\222S\177\000", __u6_addr16 = {33920, 51304, 32767, 0, 18168, 37474, 32595, 0}, __u6_addr32 = {3362292864, 32767, 2455914232, 32595}}}, sin6_scope_id = 1}}, id = 0, proto = -12 '\364', send_flags = {f = 11 '\v', blst_imask = 139 '\213'}} backup_user_from = 0x8c0bf0 backup_user_to = 0x8c0bf8 backup_domain_from = 0x8c0c00 backup_domain_to = 0x8c0c08 backup_uri_from = 0x8c0be0 ---Type <return> to continue, or q <return> to quit--- backup_uri_to = 0x8c0be8 backup_xavps = 0x8c0cf0 replies_locked = <value optimized out> branch_ret = <value optimized out> blst_503_timeout = <value optimized out> hf = <value optimized out> onsend_params = {req = 0x8b0bf4, rpl = 0x7f539260e170, param = 0x8b0d4d, code = 5583256, flags = 0, branch = 0, t_rbuf = 0x8b0e5b, dst = 0x0, send_buf = {s = 0x30 <Address 0x30 out of bounds>, len = -1839095808}} ctx = {rec_lev = 3, run_flags = 0, last_retcode = 1, jmp_env = {{__jmpbuf = {140736555680304, 4105241171855655996, 139996914874480, 139996914837568, 100, 139996784187664, -4105259996694723524, 4105241735827276860}, __mask_was_saved = 0, __saved_mask = {__val = {9113494, 9113174, 5435765, 29110320, 139996919130444, 5873181, 139996822528191, 139996784052600, 139996914927352, 17179869186, 139998502395564, 139996914446352, 5424982, 29230000, 139996885707245, 139996914446352}}}}} #8 0x0000000000453325 in forward_reply (msg=0x7f539260e840) at forward.c:790 new_buf = <value optimized out> dst = {send_sock = 0x0, to = {s = {sa_family = 0, sa_data = '\000' <repeats 13 times>}, sin = {sin_family = 0, sin_port = 0, sin_addr = {s_addr = 0}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 0, sin6_port = 0, sin6_flowinfo = 0, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, id = 0, proto = 0 '\000', send_flags = {f = 0 '\000', blst_imask = 0 '\000'}} new_len = <value optimized out> r = -1970851388 s = <value optimized out> len = <value optimized out> #9 0x0000000000497175 in receive_msg ( buf=0x8b0ba0 "SIP/2.0 100 Giving a try\r\nVia: SIP/2.0/UDP 172.30.1.76;branch=z9hG4bK3568.03a3e98.0;i=1\r\nVia: SIP/2.0/tls 172.30.1.241:54666;rport=54666;branch=z9hG4bKPj0n0hVs50Gyl6TfXJPcSI4xeJnWZ3QUKc\r\nFrom: "Samuel"..., len=<value optimized out>, rcv_info=<value optimized out>) at receive.c:270 msg = 0x7f539260e840 ctx = {rec_lev = 0, run_flags = 0, last_retcode = 0, jmp_env = {{__jmpbuf = {139996914552432, 29123520, 139996918690424, 139996928671744, 7, 4294967295, 7, 1}, __mask_was_saved = 8416480, __saved_mask = {__val = {0, 140736555681576, 28, 0, 139996926547730, 139994459013121, 0, 139996783012124, 139996918690424, 0, 139996914883664, 139996928669352, 139996914883672, 4294967245, 139996926572069, 140736555681462}}}}} ret = 0 inb = { s = 0x8b0ba0 "SIP/2.0 100 Giving a try\r\nVia: SIP/2.0/UDP 172.30.1.76;branch=z9hG4bK3568.03a3e98.0;i=1\r\nVia: SIP/2.0/tls 172.30.1.241:54666;rport=54666;branch=z9hG4bKPj0n0hVs50Gyl6TfXJPcSI4xeJnWZ3QUKc\r\nFrom: "Samuel"..., len = 429} #10 0x00000000005204b7 in udp_rcv_loop () at udp_server.c:544 len = 429 buf = "SIP/2.0 100 Giving a try\r\nVia: SIP/2.0/UDP 172.30.1.76;branch=z9hG4bK3568.03a3e98.0;i=1\r\nVia: SIP/2.0/tls 172.30.1.241:54666;rport=54666;branch=z9hG4bKPj0n0hVs50Gyl6TfXJPcSI4xeJnWZ3QUKc\r\nFrom: "Samuel"... from = 0x7f5392619c50 fromlen = 16 ri = {src_ip = {af = 2, len = 4, u = {addrl = {1459691180, 0}, addr32 = {1459691180, 0, 0, 0}, addr16 = {7852, 22273, 0, 0, 0, 0, 0, 0}, addr = "\254\036\001W", '\000' <repeats 11 times>}}, dst_ip = {af = 2, len = 4, u = {addrl = {1275141804, 0}, addr32 = {1275141804, 0, 0, 0}, addr16 = {7852, 19457, 0, 0, 0, 0, 0, 0}, addr = "\254\036\001L", '\000' <repeats 11 times>}}, src_port = 5060, dst_port = 5060, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023Ĭ\036\001W\000\000\000\000\000\000\000"}, sin = { sin_family = 2, sin_port = 50195, sin_addr = {s_addr = 1459691180}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = { sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 1459691180, sin6_addr = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, bind_address = 0x7f5392619fd0, proto = 1 '\001'} ---Type <return> to continue, or q <return> to quit--- #11 0x000000000046349d in main_loop () at main.c:1633 i = 2 pid = 0 si = 0x7f5392619fd0 si_desc = "udp receiver child=2 sock=172.30.1.76:5060 \000\000\000\000\000\000@ \360\\222S\177\000\000\005\000\000\000\t\000\000\000\030\061\275\214S\177\000\000\001\000\000\000\000\000\000\000\300j\214\000\000\000\000\000\000\000@\002", '\000' <repeats 12 times>, "\005\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000W\J\000\000\000\000" nrprocs = 4 #12 0x000000000046607c in main (argc=<value optimized out>, argv=0x0) at main.c:2546 cfg_stream = <value optimized out> c = <value optimized out> r = 0 tmp = 0x41597b "H\203\304\b\303\377\065:\017?" tmp_len = 0 port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 2037467145 rfd = <value optimized out> debug_save = <value optimized out> debug_flag = 37748736 dont_fork_cnt = 0 n_lst = 0x4 p = <value optimized out>
On 9 October 2012 12:53, Daniel-Constantin Mierla miconda@gmail.com wrote:
If you provided -w parameter, then it is at that path. If not, it should be in '/' or in the OS-configured corefiles directory.
Daniel
On 10/9/12 12:45 PM, samuel wrote:
I can not locate where the core is (wasn'i it in /tmp?) but I see the following statement in the log:
Oct 9 12:44:21 kamailio1 kamailio[17420]: ALERT: <core> [main.c:788]: core was generated
so i guess somewhere in the disk there is a core file....
Thanks a lot, Daniel, Samuel.
On 9 October 2012 12:08, Daniel-Constantin Mierla miconda@gmail.comwrote:
Did you get a core dump, can you send the backtrace?
Daniel
On 10/9/12 11:00 AM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
-- Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
The line matches the access to the action, can you send also the output inside gdb for command:
p *a
Also, as I understand you can reproduce it, can you load debugger module, set its parameter cfg_trace to 1 and run again. Then send the lines printed by the cfg trace before the crash, so I can track what actions in the config were executed.
Daniel
On 10/9/12 1:01 PM, samuel wrote:
This is the output of the core file:
(gdb) bt full #0 do_action (h=<value optimized out>, a=0x485f3832315f4d43, msg=<value optimized out>) at action.c:340
The output of the requested command is: (gdb) p *a Cannot access memory at address 0x485f3832315f4d43
I'll try to load the debugger module, and send the requested information to digg into the problem.
Thanks again! SAmuel.
On 9 October 2012 13:25, Daniel-Constantin Mierla miconda@gmail.com wrote:
The line matches the access to the action, can you send also the output inside gdb for command:
p *a
Also, as I understand you can reproduce it, can you load debugger module, set its parameter cfg_trace to 1 and run again. Then send the lines printed by the cfg trace before the crash, so I can track what actions in the config were executed.
Daniel
On 10/9/12 1:01 PM, samuel wrote:
This is the output of the core file:
(gdb) bt full #0 do_action (h=<value optimized out>, a=0x485f3832315f4d43, msg=<value optimized out>) at action.c:340
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/**micondahttp://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
These are the lines printed, as you can see there isn't yet a rtpproxy running but it has been working in the tests without crashing. The config is pretty standard so you'll see the usual routes. I've printed the line number for an easy debugging. I'm not 100% sure that it always happens in the same point. I'll try to reproduce it and see whether following errors happen in the same point of the config file.
Oct 9 15:16:25 kamailio1 kamailio[18015]: WARNING: rtpproxy [rtpproxy.c:1393]: can't get version of the RTP proxy Oct 9 15:16:25 kamailio1 kamailio[18015]: WARNING: rtpproxy [rtpproxy.c:1430]: support for RTP proxy udp:127.0.0.1:7722 has been disabled temporarily Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: rtpproxy [rtpproxy.c:2231]: no available proxies Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=832 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=827 a=25 n=is_request Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=831 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=828 a=25 n=has_totag Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=829 a=26 n=add_rr_param Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=838 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=832 a=25 n=is_reply Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=838 a=3 n=return Oct 9 15:16:25 kamailio1 kamailio[18015]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=971 a=3 n=exit Oct 9 15:16:25 kamailio1 kernel: [6556004.157495] kamailio[18015] general protection ip:532b74 sp:7fffbd33ded0 error:0 in kamailio[400000+207000] Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=988 a=26 n=xdbg Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=991 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=990 a=6 n=route Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=822 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=815 a=25 n=is_request Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=825 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=822 a=42 n=isflagset Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=825 a=25 n=rtpproxy_manage Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=832 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=827 a=25 n=is_request Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=838 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=832 a=25 n=is_reply Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=836 a=17 n=if Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=833 a=26 n=isbflagset Oct 9 15:16:25 kamailio1 kamailio[18006]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=838 a=3 n=return Oct 9 15:16:26 kamailio1 kamailio[18019]: : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 29 Oct 9 15:16:26 kamailio1 kamailio[18001]: ALERT: <core> [main.c:785]: child process 18015 exited by a signal 11 Oct 9 15:16:26 kamailio1 kamailio[18001]: ALERT: <core> [main.c:788]: core was generated
route[NATMANAGE] { #!ifdef WITH_NAT if (is_request()) { if(has_totag()) { if(check_route_param("nat=yes")) { setbflag(FLB_NATB); } } } if (!(isflagset(FLT_NATS) || isbflagset(FLB_NATB))) return;
rtpproxy_manage();
827: if (is_request()) { if (!has_totag()) { add_rr_param(";nat=yes"); } 831: } if (is_reply()) { if(isbflagset(FLB_NATB)) { fix_nated_contact(); } } #!endif 838: return; }
959:route[RELAY_TO_PSTN] {
# enable additional event routes for forwarded requests # - serial forking, RTP relaying handling, a.s.o. if (is_method("INVITE|SUBSCRIBE")) { t_on_branch("MANAGE_BRANCH"); t_on_reply("MANAGE_REPLY"); }
if (!t_relay()) { sl_reply_error(); } 971: exit; }
route[HOMER] { setflag(22); sip_trace(); }
# manage outgoing branches branch_route[MANAGE_BRANCH] { xdbg("new branch [$T_branch_idx] to $ru\n"); route(NATMANAGE); }
# manage incoming replies onreply_route[MANAGE_REPLY] { 988: xdbg("incoming reply\n"); if(status=~"[12][0-9][0-9]") route(NATMANAGE); 991:}
On 9 October 2012 15:11, samuel samu60@gmail.com wrote:
The output of the requested command is: (gdb) p *a Cannot access memory at address 0x485f3832315f4d43
I'll try to load the debugger module, and send the requested information to digg into the problem.
Thanks again! SAmuel.
On 9 October 2012 13:25, Daniel-Constantin Mierla miconda@gmail.comwrote:
The line matches the access to the action, can you send also the output inside gdb for command:
p *a
Also, as I understand you can reproduce it, can you load debugger module, set its parameter cfg_trace to 1 and run again. Then send the lines printed by the cfg trace before the crash, so I can track what actions in the config were executed.
Daniel
On 10/9/12 1:01 PM, samuel wrote:
This is the output of the core file:
(gdb) bt full #0 do_action (h=<value optimized out>, a=0x485f3832315f4d43, msg=<value optimized out>) at action.c:340
-- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/**micondahttp://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
As said in the previous mail, the problem is not always in the same point of the config. I just created a second crash: Oct 9 15:30:13 kamailio1 kamailio[18152]: INFO: <core> [mem/f_malloc.c:529]: freeing a free fragment (0x7fd66029c2f8/0x7fd66029c308) - ignore Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=520 a=6 n=route Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=667 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=632 a=25 n=has_totag Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=664 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=635 a=25 n=loose_route Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=651 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=646 a=26 n=is_method Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=651 a=26 n=is_method Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply Oct 9 15:30:13 kamailio1 kernel: [6556831.473688] kamailio[18152] general protection ip:47b9d5 sp:7fffab3a22d0 error:0 in kamailio[400000+207000] Oct 9 15:30:13 kamailio1 kamailio[18155]: : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 30 Oct 9 15:30:13 kamailio1 kamailio[18137]: ALERT: <core> [main.c:785]: child process 18152 exited by a signal 11 Oct 9 15:30:13 kamailio1 kamailio[18137]: ALERT: <core> [main.c:788]: core was generated
last lines 6xx are the usual handling of request within dialog, being the last one a sl_send_reply.
In this case, the ouput of the full backtrace is: Core was generated by `kamailio'. Program terminated with signal 11, Segmentation fault. #0 build_res_buf_from_sip_req (code=<value optimized out>, text=<value optimized out>, new_tag=0x0, msg=0x7fd6602a17d0, returned_len=<value optimized out>, bmark=<value optimized out>) at msg_translator.c:2028 2028 switch (hdr->type) { (gdb) bt full #0 build_res_buf_from_sip_req (code=<value optimized out>, text=<value optimized out>, new_tag=0x0, msg=0x7fd6602a17d0, returned_len=<value optimized out>, bmark=<value optimized out>) at msg_translator.c:2028 buf = <value optimized out> p = <value optimized out> len = 3365587166 hdr = 0x73550a0d75757267 lump = <value optimized out> body = <value optimized out> received_buf = <value optimized out> received_len = 0 rport_buf = <value optimized out> rport_len = 12 warning_buf = <value optimized out> warning_len = 0 content_len_buf = <value optimized out> content_len_len = 0 after_body = <value optimized out> totags = <value optimized out> #1 0x00007fd65dc193ac in sl_reply_helper (msg=0x7fd6602a17d0, code=404, reason=0x7fd66029b1b0 "Not here", tag=0x0) at sl_funcs.c:181 buf = {s = 0x0, len = 0} ---Type <return> to continue, or q <return> to quit--- dset = {s = 0x0, len = 0} dst = {send_sock = 0x0, to = {s = {sa_family = 2, sa_data = "\333\000\254\036\001\361\000\000\000\000\000\000\000"}, sin = {sin_family = 2, sin_port = 219, sin_addr = {s_addr = 4043382444}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = { sin6_family = 2, sin6_port = 219, sin6_flowinfo = 4043382444, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}, id = 0, proto = 0 '\000', send_flags = {f = 0 '\000', blst_imask = 0 '\000'}} dummy_bm = {to_tag_val = {s = 0xec0030 "CEST", len = 0}} backup_mhomed = 1617693440 ret = <value optimized out> text = {s = 0x7fd66029b1b0 "Not here", len = 8} rt = <value optimized out> backup_rt = <value optimized out> pmsg = {id = 2872715568, pid = 32767, tval = { tv_sec = 140558717391654, tv_usec = 15640528}, fwd_send_flags = { f = 240 '\360', blst_imask = 40 '('}, rpl_send_flags = { f = 58 ':', blst_imask = 171 '\253'}, first_line = { type = 1613371344, len = 32726, u = {request = {method = { s = 0x7fffab3a2bd0 "0", len = 1467957462}, uri = { s = 0x7fd660770309 "T ", len = 15641148}, version = { ---Type <return> to continue, or q <return> to quit--- s = 0x7fd6609a33e0 " -\232`\326\177", len = 15641148}, method_value = 7}, reply = {version = {s = 0x7fffab3a2bd0 "0", len = 1467957462}, status = {s = 0x7fd660770309 "T ", len = 15641148}, reason = { s = 0x7fd6609a33e0 " -\232`\326\177", len = 15641148}, statuscode = 7}}}, via1 = 0x1ffb, via2 = 0x7fd6606e1289, headers = 0x353236d6609a6e40, last_header = 0x0, parsed_flag = 140736066102597, h_via1 = 0x7fd600000000, h_via2 = 0xab3a2545, callid = 0x1, to = 0x3000000010, cseq = 0x7fffab3a2790, from = 0x7fffab3a26d0, contact = 0x0, maxforwards = 0xffffffffffffffa8, route = 0xeeaa20, record_route = 0x2010, content_type = 0xeeca30, content_length = 0x7fd6609a6e40, authorization = 0x90, expires = 0x2010, proxy_auth = 0x7fd6606c0513, supported = 0x7fffab3a2580, require = 0x7fd6609a6e40, proxy_require = 0xeeaa30, unsupported = 0x7e, allow = 0xeeaa20, event = 0x90, accept = 0x2010, accept_language = 0x7fd6606c0b00, organization = 0x0, priority = 0xeea7d0, subject = 0x7fffab3a2660, user_agent = 0x7fd6609a6e40, server = 0x0, content_disposition = 0xffffffff, diversion = 0x0, rpid = 0x7fd6609a6e40, refer_to = 0x15, session_expires = 0xffffffff, min_se = 0x0, sipifmatch = 0xffffffffffffffa8, ---Type <return> to continue, or q <return> to quit--- subscription_state = 0x7fffab3a26b0, date = 0x7fd6606bf94c, identity = 0xeea7d0, identity_info = 0x7fd660714c6d, pai = 0x7fd66029c388, ppi = 0x7fd6577f40b8, path = 0x1e0000000d, privacy = 0x90000000f, body = 0x7000000009, eoh = 0x11a00000002 <Address 0x11a00000002 out of bounds>, unparsed = 0x1 <Address 0x1 out of bounds>, rcv = {src_ip = { af = 7200, len = 0, u = {addrl = {15466544, 140558712886192}, addr32 = {15466544, 0, 1613157296, 32726}, addr16 = {48, 236, 0, 0, 54192, 24614, 32726, 0}, addr = "0\000\354\000\000\000\000\000\260\323&`\326\177\000"}}, dst_ip = {af = 15640528, len = 0, u = {addrl = {140558717559926, 6}, addr32 = { 1617831030, 32726, 6, 0}, addr16 = {9334, 24686, 32726, 0, 6, 0, 0, 0}, addr = "v$n`\326\177\000\000\006\000\000\000\000\000\000"}}, src_port = 11472, dst_port = 43834, proto_reserved1 = 32767, proto_reserved2 = 1618412303, src_su = {s = {sa_family = 32726, sa_data = "\000\000\270\262h`\326\177\000\000\001\000\000"}, sin = {sin_family = 32726, sin_port = 0, sin_addr = { s_addr = 1617474232}, sin_zero = "\326\177\000\000\001\000\000"}, sin6 = { sin6_family = 32726, sin6_port = 0, sin6_flowinfo = 1617474232, sin6_addr = {__in6_u = { __u6_addr8 = "\326\177\000\000\001\000\000\000\000\000\000\0---Type <return> to continue, or q <return> to quit--- 00\002\000\000", __u6_addr16 = {32726, 0, 1, 0, 0, 0, 2, 0}, __u6_addr32 = { 32726, 1, 0, 2}}}, sin6_scope_id = 0}}, bind_address = 0x7fffab3a28a0, proto = 0 '\000'}, buf = 0x7fffab3a26f6 "", len = 0, new_uri = { s = 0x32353138313a26e0 <Address 0x32353138313a26e0 out of bounds>, len = -1422252298}, dst_uri = { s = 0x7fd6609a3020 "\322\372v`\326\177", len = -1422251905}, parsed_uri_ok = 15466544, parsed_uri = {user = { s = 0x7fffab3a26a0 "\320':\253\377\177", len = 16}, passwd = { s = 0x7fffab3a2890 "\320\027*`\326\177", len = -1422252080}, host = {s = 0x7fd66027d350 "\206\002", len = 0}, port = { s = 0x7fffab3a27a0 "\020 ", len = 5924323}, params = { s = 0x59773d "corefunc", len = 8426208}, sip_params = { s = 0x41e406 "L\213\r\033\216>", len = 5873181}, headers = { s = 0x7fd6577f3f85 "*** cfgtrace:", len = 5909272}, port_no = 635, proto = 0, type = ERROR_URI_T, flags = 1467957462, transport = {s = 0x7fd660770309 "T ", len = 15641148}, ttl = { s = 0x7fd6609a33e0 " -\232`\326\177", len = 15641148}, user_param = {s = 0x7 <Address 0x7 out of bounds>, len = 8187}, maddr = {s = 0x7fd6606e1289 "HcD$@D\213\$\bH\215D\005", len = 1613371344}, method = { s = 0x7fff00000000 <Address 0x7fff00000000 out of bounds>, len = -1422251995}, lr = {s = 0x0, len = -1422251995}, r2 = { ---Type <return> to continue, or q <return> to quit--- s = 0x1 <Address 0x1 out of bounds>, len = 16}, gr = { s = 0x7fffab3a2a70 "\033", len = -1422251600}, transport_val = { s = 0x0, len = -88}, ttl_val = {s = 0xeeaa20 "`\002", len = 8208}, user_param_val = {s = 0xeeca30 "", len = 1620733504}, maddr_val = { s = 0x90 <Address 0x90 out of bounds>, len = 8208}, method_val = { s = 0x7fd6606c0513 "H\215C\020H\213\$\bH\213l$\020L\213d$\030L\213l$ L\213t$(L\213|$0H\203\304\070\303f\017\037D", len = -1422251936}, lr_val = {s = 0x7fd6609a6e40 "", len = 15641136}, r2_val = { s = 0x80 <Address 0x80 out of bounds>, len = 15641120}, gr_val = {s = 0x90 <Address 0x90 out of bounds>, len = 8208}}, parsed_orig_ruri_ok = 1617693440, parsed_orig_ruri = {user = { s = 0x0, len = 15640528}, passwd = { s = 0x7fffab3a2940 "0\252", <incomplete sequence \356>, len = 1620733504}, host = {s = 0x0, len = -1}, port = {s = 0x0, len = 1620733504}, params = { s = 0x15 <Address 0x15 out of bounds>, len = -1}, sip_params = { s = 0x0, len = -88}, headers = {s = 0x7fffab3a2990 "0", len = 1617688908}, port_no = 42960, proto = 238, type = ERROR_URI_T, flags = 1618037869, transport = { s = 0x7fd6602a17d0 "\f", len = 1467957432}, ttl = { s = 0x1e0000000d <Address 0x1e0000000d out of bounds>, ---Type <return> to continue, or q <return> to quit--- len = 15}, user_param = { s = 0x7000000009 <Address 0x7000000009 out of bounds>, len = 2}, maddr = {s = 0x1 <Address 0x1 out of bounds>, len = 7200}, method = {s = 0xec0030 "CEST", len = 5863229}, lr = { s = 0x8092e0 "1\350Y", len = 4319780}, r2 = {s = 0x0, len = -1}, gr = {s = 0x7fffab3a4050 "\004", len = 195287850}, transport_val = {s = 0x7fffab3a4050 "\004", len = -1422250008}, ttl_val = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 0}, user_param_val = { s = 0x4598de195bc3db2a <Address 0x4598de195bc3db2a out of bounds>, len = 127}, maddr_val = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 1349789413}, method_val = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 8443048}, lr_val = {s = 0x7fd660280420 "\227\002", len = 8474768}, r2_val = { s = 0x7fd6579f5cf0 "\273\363\301]\326\177", len = 1481687540}, gr_val = {s = 0x807228 "\364\301PX\326\177", len = 1618038976}}, add_rm = 0x3000000030, body_lumps = 0x7fffab3a2a88, ---Type <return> to continue, or q <return> to quit--- reply_lump = 0x7fffab3a29b0, add_to_branch_s = "\001\000\000\000\000\000\000\000\020\000\000\000\060\000\000\000\260,:\253\377\177\000\000\035\236Y\000\000\000\000\000\205?\177W\326\177\000\000\030+Z\000\000\000\000\000\227\002\000\000\000\000\000\000P@",
add_to_branch_len = 32767, hash_index = 4931461, msg_flags = 0, flags = 1620733504, set_global_address = { s = 0x90 <Address 0x90 out of bounds>, len = 8208}, set_global_port = { s = 0x7fd6606c0513 "H\215C\020H\213\$\bH\213l$\020L\213d$\030L\213l$ L\213t$(L\213|$0H\203\304\070\303f\017\037D", len = -1422251360}, force_send_socket = 0x7fd6609a6e40, path_vec = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 124}} #2 0x00007fd65dc17348 in w_sl_send_reply (msg=0x7fd6602a17d0, p1=<value optimized out>, p2=<value optimized out>) at sl.c:221 code = 404 ret = 1970631271 reason = {s = 0x7fd6602805a0 "Not here", len = 8} r = 0x7fd66029b1b0 "Not here" #3 0x0000000000418561 in do_action (h=<value optimized out>, a=0x7fd660280420, msg=<value optimized out>) at action.c:1157 ret = -5 ---Type <return> to continue, or q <return> to quit--- v = <value optimized out> dst = {send_sock = 0x7fffab3a3035, to = {s = {sa_family = 0, sa_data = "\000\000\326\177\000\000\065\060:\253\000\000\000"}, sin = {sin_family = 0, sin_port = 0, sin_addr = {s_addr = 32726}, sin_zero = "50:\253\000\000\000"}, sin6 = {sin6_family = 0, sin6_port = 0, sin6_flowinfo = 32726, sin6_addr = {__in6_u = { __u6_addr8 = "50:\253\000\000\000\000x\264'`\326\177\000", __u6_addr16 = {12341, 43834, 0, 0, 46200, 24615, 32726, 0}, __u6_addr32 = {2872717365, 0, 1613214840, 32726}}}, sin6_scope_id = 1613371344}}, id = 32726, proto = 80 'P', send_flags = {f = 64 '@', blst_imask = 58 ':'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0xeea7d0 "\230n\232`\326\177", len = 1618037869}, passwd = {s = 0x7fd6602a17d0 "\f", len = -1422250480}, host = { s = 0x7fd66027ab28 "{\002", len = 15}, port = { s = 0x7000000009 <Address 0x7000000009 out of bounds>, len = -1422245808}, params = {s = 0x0, len = -1422245808}, sip_params = {s = 0x7fffab3a3508 "", len = 5863229}, headers = { ---Type <return> to continue, or q <return> to quit--- s = 0x8092e0 "1\350Y", len = 4319780}, port_no = 0, proto = 0, type = ERROR_URI_T, flags = 4294967295, transport = { s = 0x7fffab3a4050 "\004", len = 195287850}, ttl = { s = 0x7fffab3a4050 "\004", len = -1422248696}, user_param = { s = 0x59773d "corefunc", len = 8426208}, maddr = { s = 0x4598de195103db2a <Address 0x4598de195103db2a out of bounds>, len = -612902102}, method = {s = 0x0, len = 1349789413}, lr = { s = 0x32353138313a2ef0 <Address 0x32353138313a2ef0 out of bounds>, len = 8443048}, r2 = {s = 0x7fd6602805c0 "\230\002", len = 8474768}, gr = {s = 0x7fd6579f5530 "\t<\177W\326\177", len = 1481687540}, transport_val = { s = 0x807228 "\364\301PX\326\177", len = 1618038976}, ttl_val = { s = 0x3000000030 <Address 0x3000000030 out of bounds>, len = -1422250072}, user_param_val = {s = 0x7fffab3a2ed0 "\020", len = 5424982}, maddr_val = { s = 0x3000000010 <Address 0x3000000010 out of bounds>, len = -1422249520}, method_val = {s = 0x599e1d "ERROR", len = 1467957125}, lr_val = { s = 0x5a2b18 "/usr/local/etc/kamailio/kamailio.cfg", len = -1422248696}, r2_val = {s = 0x7fffab3a4050 "\004", len = 4931461}, gr_val = {s = 0x7fffab3a3508 "", len = 1618412297}} next_hop = {user = {s = 0x7fd6577f40d6 "\n", len = 1618412297}, passwd = { ---Type <return> to continue, or q <return> to quit--- s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 1620718560}, host = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 7}, port = {s = 0x1ffb <Address 0x1ffb out of bounds>, len = 1617826441}, params = { s = 0x373136ffab3a31b0 <Address 0x373136ffab3a31b0 out of bounds>, len = 0}, sip_params = {s = 0x7fffab3a2d45 "", len = 0}, headers = {s = 0xab3a2d45 <Address 0xab3a2d45 out of bounds>, len = 1}, port_no = 16, proto = 0, type = 48, flags = 2872717200, transport = {s = 0x7fffab3a2ed0 "\020", len = 0}, ttl = { s = 0xffffffffffffffa8 <Address 0xffffffffffffffa8 out of bounds>, len = 15641120}, user_param = { s = 0x2010 <Address 0x2010 out of bounds>, len = 15649328}, maddr = {s = 0x7fd66027ab28 "{\002", len = 1613371344}, method = { s = 0x2010 <Address 0x2010 out of bounds>, len = 1617691923}, lr = {s = 0x7fffab3a2d80 "", len = 1620733504}, r2 = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 117}, gr = {s = 0xeeaa20 "`\002", len = 1613212456}, transport_val = {s = 0x0, len = -1422250480}, ttl_val = { s = 0x5a65e3 "", len = 5863229}, user_param_val = { ---Type <return> to continue, or q <return> to quit--- s = 0x8092e0 "1\350Y", len = 4318214}, maddr_val = {s = 0x0, len = -1}, method_val = {s = 0x0, len = 1620733504}, lr_val = { s = 0x15 <Address 0x15 out of bounds>, len = -1}, r2_val = { s = 0x0, len = -88}, gr_val = {s = 0x7fffab3a2eb0 "0", len = 1617688908}} u = <value optimized out> port = <value optimized out> i = <value optimized out> flags = <value optimized out> avp = <value optimized out> st = {flags = 2872718600, id = 32767, name = {n = 4948316, s = { s = 0x4b815c "\205\300A\211\307\017\211u\376\377\377\350\004\323\370\377\205\300f\220\017\210\272\005", len = 1467956233}, re = 0x4b815c}, avp = 0xeeaa20} sct = <value optimized out> sjt = <value optimized out> rve = <value optimized out> mct = <value optimized out> rv = <value optimized out> rv1 = <value optimized out> c1 = {cache_type = 15641148, val_type = RV_NONE, c = {avp_val = { n = 1620718560, s = {s = 0x7fd6609a33e0 " -\232`\326\177", len = 15641148}, re = 0x7fd6609a33e0}, pval = {rs = { ---Type <return> to continue, or q <return> to quit--- s = 0x7fd6609a33e0 " -\232`\326\177", len = 15641148}, ri = 7, flags = 0}}, i2s = "\373\037\000\000\000\000\000\000\211\022n`7fd66029c3"} s = {s = 0x7fd6609a6e40 "", len = 160} srevp = {0x7fd660280420, 0x7fd6602a17d0} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #4 0x000000000041e406 in run_actions (h=0x7fffab3a4050, ---Type <return> to continue, or q <return> to quit--- a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7fd660280420 ret = 1 mod = <value optimized out> ms = 1 #5 0x0000000000419c08 in do_action (h=<value optimized out>, a=0x7fd6602805c0, msg=<value optimized out>) at action.c:1140 ret = 1 v = 0 dst = {send_sock = 0x8092e0, to = {s = {sa_family = 4700, sa_data = "\240W\326\177\000\000\320\027*`\326\177\000"}, sin = { sin_family = 4700, sin_port = 22432, sin_addr = { s_addr = 32726}, sin_zero = "\320\027*`\326\177\000"}, sin6 = { sin6_family = 4700, sin6_port = 22432, sin6_flowinfo = 32726, sin6_addr = {__in6_u = { __u6_addr8 = "\320\027*`\326\177\000\000\250\255'`\326\177\000", __u6_addr16 = {6096, 24618, 32726, 0, 44456, 24615, 32726, 0}, __u6_addr32 = {1613371344, 32726, 1613213096, 32726}}}, sin6_scope_id = 1613371344}}, id = 32726, proto = 80 'P', send_flags = {f = 64 '@', blst_imask = 58 ':'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> ---Type <return> to continue, or q <return> to quit--- crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0xeea7d0 "\230n\232`\326\177", len = 1618037869}, passwd = {s = 0x7fd6602a17d0 "\f", len = -1422249168}, host = { s = 0x7fd66027a9a8 "x\002", len = 15}, port = { s = 0x7000000009 <Address 0x7000000009 out of bounds>, len = -1422245808}, params = {s = 0x0, len = -1422245808}, sip_params = {s = 0x7fffab3a3a28 "\001", len = 5863229}, headers = { s = 0x8092e0 "1\350Y", len = 4319780}, port_no = 0, proto = 0, type = ERROR_URI_T, flags = URI_USER_NORMALIZE, transport = { s = 0x7fffab3a4050 "\004", len = 195287850}, ttl = { s = 0x7fffab3a4050 "\004", len = -1422247384}, user_param = { s = 0x59773d "corefunc", len = 8426208}, maddr = { s = 0x4598de196f43db2a <Address 0x4598de196f43db2a out of bounds>, len = -612902102}, method = {s = 0x0, len = 1349789413}, lr = { s = 0xedd9c3 "", len = 8443048}, r2 = { s = 0x7fd6602807a0 "\233\002", len = 8474768}, gr = { s = 0x7fd6579f5530 "\t<\177W\326\177", len = 1481687540}, transport_val = {s = 0x807228 "\364\301PX\326\177", len = 1618038976}, ttl_val = { s = 0x3000000030 <Address 0x3000000030 out of bounds>, len = -1422248760}, user_param_val = { ---Type <return> to continue, or q <return> to quit--- s = 0x7fffab3a33f0 "\345&tP", len = 0}, maddr_val = { s = 0x507426e5 <Address 0x507426e5 out of bounds>, len = 268346}, method_val = {s = 0x599e1d "ERROR", len = 1467957125}, lr_val = { s = 0x5a2b18 "/usr/local/etc/kamailio/kamailio.cfg", len = -1422247384}, r2_val = {s = 0x7fffab3a4050 "\004", len = 4931461}, gr_val = { s = 0xc51300db16021002 <Address 0xc51300db16021002 out of bounds>, len = 1613371344}} next_hop = {user = {s = 0x7fd6577f40d6 "\n", len = 1618412297}, passwd = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 1620718560}, host = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 7}, port = {s = 0x1ffb <Address 0x1ffb out of bounds>, len = 1617826441}, params = { s = 0x373136d6609a6e40 <Address 0x373136d6609a6e40 out of bounds>, len = 0}, sip_params = {s = 0x7fffab3a3265 "", len = 0}, headers = {s = 0xab3a3265 <Address 0xab3a3265 out of bounds>, len = 1}, port_no = 16, proto = 0, type = 48, flags = 2872718512, transport = {s = 0x7fffab3a33f0 "\345&tP", len = 0}, ttl = { s = 0xffffffffffffffa8 <Address 0xffffffffffffffa8 out of bounds>, ---Type <return> to continue, or q <return> to quit--- len = 15641120}, user_param = { s = 0x2010 <Address 0x2010 out of bounds>, len = 15649328}, maddr = {s = 0x7fd66027a9a8 "x\002", len = 1613371344}, method = { s = 0x2010 <Address 0x2010 out of bounds>, len = 1617691923}, lr = {s = 0x7fffab3a32a0 "", len = 1620733504}, r2 = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 117}, gr = {s = 0xeeaa20 "`\002", len = 1613212072}, transport_val = {s = 0x0, len = -1422249168}, ttl_val = { s = 0x5a65e3 "", len = 5863229}, user_param_val = { s = 0x8092e0 "1\350Y", len = 4318214}, maddr_val = {s = 0x0, len = -1}, method_val = {s = 0x0, len = 1620733504}, lr_val = { s = 0x15 <Address 0x15 out of bounds>, len = -1}, r2_val = { s = 0x0, len = -88}, gr_val = {s = 0x7fffab3a33d0 "0", len = 1617688908}} u = <value optimized out> port = <value optimized out> i = <value optimized out> flags = <value optimized out> avp = <value optimized out> st = {flags = 2872719912, id = 32767, name = {n = 4948316, s = { s = 0x4b815c "\205\300A\211\307\017\211u\376\377\377\350\004\323\370\377\205\300f\220\017\210\272\005", len = 1467956233}, re = 0x4b815c}, ---Type <return> to continue, or q <return> to quit--- avp = 0x0} sct = <value optimized out> sjt = <value optimized out> rve = 0x7fd66027b478 mct = <value optimized out> rv = <value optimized out> rv1 = <value optimized out> c1 = {cache_type = RV_CACHE_EMPTY, val_type = RV_NONE, c = {avp_val = { n = -1422248784, s = { s = 0x16ab3a34b0 <Address 0x16ab3a34b0 out of bounds>, len = 1472248392}, re = 0x16ab3a34b0}, pval = {rs = { s = 0x16ab3a34b0 <Address 0x16ab3a34b0 out of bounds>, len = 1472248392}, ri = 1613371344, flags = 32726}}, i2s = " \276\300W\326\177\000\000\260\064:\253\377\177\000\000\215\204\240W\326\177"} s = {s = 0x0, len = 1482622128} srevp = {0x7fd6602805c0, 0x7fd6602a17d0} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, ---Type <return> to continue, or q <return> to quit--- select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #6 0x000000000041e406 in run_actions (h=0x7fffab3a4050, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7fd6602805c0 ret = 0 mod = <value optimized out> ms = 0 #7 0x000000000041779e in do_action (h=<value optimized out>, a=0x7fd6602807a0, msg=<value optimized out>) at action.c:1136 ret = 1 v = 1 dst = {send_sock = 0x30, to = {s = {sa_family = 23560, sa_data = "\006^\326\177\000\000\060\377&`\326\177\000"}, sin = { ---Type <return> to continue, or q <return> to quit--- sin_family = 23560, sin_port = 24070, sin_addr = { s_addr = 32726}, sin_zero = "0\377&`\326\177\000"}, sin6 = { sin6_family = 23560, sin6_port = 24070, sin6_flowinfo = 32726, sin6_addr = {__in6_u = { __u6_addr8 = "0\377&`\326\177\000\000\250\265&`\326\177\000", __u6_addr16 = {65328, 24614, 32726, 0, 46504, 24614, 32726, 0}, __u6_addr32 = { 1613168432, 32726, 1613149608, 32726}}}, sin6_scope_id = 1613371344}}, id = 32726, proto = -41 '\327', send_flags = {f = 27 '\033', blst_imask = 127 '\177'}} tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = {s = 0xeea7d0 "\230n\232`\326\177", len = 1618037869}, passwd = {s = 0x7fd6602a17d0 "\f", len = 1467957432}, host = { s = 0x1e0000000d <Address 0x1e0000000d out of bounds>, len = 15}, port = {s = 0x7000000009 <Address 0x7000000009 out of bounds>, len = 2}, params = {s = 0x1 <Address 0x1 out of bounds>, len = 7200}, sip_params = {s = 0xec0030 "CEST", len = 5863229}, headers = {s = 0x8092e0 "1\350Y", len = 4319780}, port_no = 0, proto = 0, type = ERROR_URI_T, flags = 4294967295, transport = { ---Type <return> to continue, or q <return> to quit--- s = 0x7fffab3a4050 "\004", len = 195287850}, ttl = { s = 0x7fffab3a4050 "\004", len = -1422246072}, user_param = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 0}, maddr = { s = 0x4598de196483db2a <Address 0x4598de196483db2a out of bounds>, len = 118}, method = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 1349789413}, lr = {s = 0x7fd66077030c "[%d]", len = 8443048}, r2 = {s = 0x7fd66026b2a8 "\b\002", len = 8474768}, gr = {s = 0x7fd6579f5428 "\236;\177W\326\177", len = 1481687540}, transport_val = {s = 0x807228 "\364\301PX\326\177", len = 1618038976}, ttl_val = { s = 0x3000000030 <Address 0x3000000030 out of bounds>, len = -1422247448}, user_param_val = { s = 0x7fffab3a3910 "03", <incomplete sequence \354>, len = 1577470454}, maddr_val = {s = 0xec3330 "\006", len = -1422245936}, method_val = {s = 0x599e1d "ERROR", len = 1467957125}, lr_val = { s = 0x5a2b18 "/usr/local/etc/kamailio/kamailio.cfg", len = 520}, r2_val = {s = 0x7fffab3a4050 "\004", len = 4931461}, gr_val = { s = 0x7fffab3a3a18 "", len = 4943585}} ---Type <return> to continue, or q <return> to quit--- next_hop = {user = {s = 0x0, len = 1618412297}, passwd = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 1620718560}, host = { s = 0xeeaa3c "15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 7}, port = {s = 0x1ffb <Address 0x1ffb out of bounds>, len = 1617826441}, params = { s = 0x363235d6609a6e40 <Address 0x363235d6609a6e40 out of bounds>, len = 0}, sip_params = {s = 0x7fffab3a3785 "\177", len = 0}, headers = {s = 0xab3a3785 <Address 0xab3a3785 out of bounds>, len = 1}, port_no = 16, proto = 0, type = 48, flags = 2872719824, transport = {s = 0x7fffab3a3910 "03", <incomplete sequence \354>, len = 0}, ttl = { s = 0xffffffffffffffa8 <Address 0xffffffffffffffa8 out of bounds>, len = 15641120}, user_param = { s = 0x2010 <Address 0x2010 out of bounds>, len = 15649328}, maddr = {s = 0x7fd6609a6e40 "", len = 128}, method = { s = 0x2010 <Address 0x2010 out of bounds>, len = 1617691923}, lr = {s = 0x7fffab3a37c0 "", len = 1620733504}, r2 = { s = 0xeeaa30 "<131>Oct 9 15:30:13 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply\n", len = 119}, gr = {s = 0xeeaa20 "`\002", len = 128}, ---Type <return> to continue, or q <return> to quit--- transport_val = {s = 0x2010 <Address 0x2010 out of bounds>, len = 1617693440}, ttl_val = {s = 0x0, len = 15640528}, user_param_val = { s = 0x7fffab3a38a0 "0\252", <incomplete sequence \356>, len = 1620733504}, maddr_val = {s = 0x0, len = -1}, method_val = { s = 0x0, len = 1620733504}, lr_val = { s = 0x15 <Address 0x15 out of bounds>, len = -1}, r2_val = { s = 0x0, len = -88}, gr_val = {s = 0x7fffab3a38f0 "0", len = 1617688908}} u = <value optimized out> port = <value optimized out> i = <value optimized out> flags = <value optimized out> avp = <value optimized out> st = {flags = 6, id = 32767, name = {n = 5, s = { s = 0x5 <Address 0x5 out of bounds>, len = 1467956126}, re = 0x5}, avp = 0x0} sct = <value optimized out> sjt = <value optimized out> rve = 0x7fd66027ada8 mct = <value optimized out> rv = <value optimized out> rv1 = <value optimized out> ---Type <return> to continue, or q <return> to quit--- c1 = {cache_type = 18152, val_type = RV_NONE, c = {avp_val = { n = 1577342303, s = {s = 0x7fd65e04555f "L\213-\n/&", len = -1422246208}, re = 0x7fd65e04555f}, pval = {rs = { s = 0x7fd65e04555f "L\213-\n/&", len = -1422246208}, ri = 1613371344, flags = 32726}}, i2s = "\322.\f", '\000' <repeats 13 times>, "6\366\260\062\000"} s = {s = 0xfbad8001 <Address 0xfbad8001 out of bounds>, len = 0} srevp = {0x7fd6602807a0, 0x7fd6602a17d0} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, ---Type <return> to continue, or q <return> to quit--- string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #8 0x000000000041e406 in run_actions (h=0x7fffab3a4050, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7fd6602807a0 ret = 0 mod = <value optimized out> ms = 0 #9 0x000000000041742c in do_action (h=<value optimized out>, a=0x7fd66026b2a8, msg=<value optimized out>) at action.c:761 ret = -5 v = <value optimized out> dst = {send_sock = 0x7fd6585f01e0, to = {s = {sa_family = 19492, sa_data = "\240W\326\177\000\000\264C:\253\377\177\000"}, sin = { sin_family = 19492, sin_port = 22432, sin_addr = { s_addr = 32726}, sin_zero = "\264C:\253\377\177\000"}, sin6 = { sin6_family = 19492, sin6_port = 22432, sin6_flowinfo = 32726, sin6_addr = {__in6_u = { __u6_addr8 = "\264C:\253\377\177\000\000\340\001_X\326\177\000", __u6_addr16 = {17332, 43834, 32767, 0, 480, 22623, 32726, 0}, __u6_addr32 = {2872722356, 32767, 1482621408, 32726}}}, sin6_scope_id = 0}}, id = 0, proto = -19 '\355', send_flags = { f = 249 '\371', blst_imask = 109 'm'}} ---Type <return> to continue, or q <return> to quit--- tmp = <value optimized out> new_uri = <value optimized out> end = <value optimized out> crt = <value optimized out> len = <value optimized out> user = <value optimized out> uri = {user = { s = 0x7fd66029b740 "\002\020\002\026", <incomplete sequence \333>, len = 1470100250}, passwd = { s = 0x7fd66064b95c "\226\201B\250\017\204\035\346|`\234|\374Qcq\zy\360\003\346`\033|\231/\310Km\301\243\353\372\266=/\222\210\353", <incomplete sequence \352>, len = -1422246256}, host = {s = 0x7fd66064cc28 "", len = 1617241520}, port = {s = 0x7fd6609a6e40 "", len = -1422246472}, params = {s = 0x0, len = 1472246968}, sip_params = {s = 0x1b3 <Address 0x1b3 out of bounds>, len = 4253884}, headers = {s = 0x8c0d00 "\020\200$`\326\177", len = -1422246496}, port_no = 0, proto = 0, type = SIP_URI_T, flags = 1584265709, transport = {s = 0x7fd660248010 "", len = 1482622431}, ttl = { s = 0x7fd66029bfa0 "ESSAGE, REFER\r\nSupported: 100rel, replaces, norefersub, gruu\r\nUser-Agent: Blink 0.2.10 (Linux)\r\nProxy-Authorization: Digest username="sam", realm="172.30.1.76", nonce="UHQoEFB0JuRjGfDQ2qkrSIM2nS3NiRnw"..., len = 1482622546}, user_param = {s = 0x7fd6602b32f0 "\b", ---Type <return> to continue, or q <return> to quit--- len = 1680015362}, maddr = {s = 0x0, len = 0}, method = {s = 0x0, len = 1}, lr = {s = 0x1 <Address 0x1 out of bounds>, len = 1482622353}, r2 = {s = 0x5a65e3 "", len = 1627275560}, gr = { s = 0x4c011eac13c50002 <Address 0x4c011eac13c50002 out of bounds>, len = 0}, transport_val = {s = 0x0, len = 0}, ttl_val = { s = 0xf1011eacdb000002 <Address 0xf1011eacdb000002 out of bounds>, len = 0}, user_param_val = {s = 0x0, len = 0}, maddr_val = { s = 0x507426e5 <Address 0x507426e5 out of bounds>, len = 216143}, method_val = { s = 0x34c4f507426e5 <Address 0x34c4f507426e5 out of bounds>, len = 5439489}, lr_val = {s = 0x0, len = 1}, r2_val = { s = 0x4c011eacf1011eac <Address 0x4c011eacf1011eac out of bounds>, len = 0}, gr_val = { s = 0xc51300db16021002 <Address 0xc51300db16021002 out of bounds>, len = 1613371344}} next_hop = {user = { s = 0x3000000018 <Address 0x3000000018 out of bounds>, len = 5366280}, passwd = {s = 0x7fffab3a3e20 "", len = 1349789241}, host = {s = 0x7fffab3a3bd0 "\001", len = 1613443480}, port = {s = 0x0, len = 1613477200}, params = { s = 0x7fd66029c340 "d#", len = 1}, sip_params = { s = 0x1 <Address 0x1 out of bounds>, len = 4457551}, headers = { s = 0x3000000018 <Address 0x3000000018 out of bounds>, ---Type <return> to continue, or q <return> to quit--- len = -1422246864}, port_no = 2360, proto = 229, type = ERROR_URI_T, flags = 2872720672, transport = { s = 0x1be <Address 0x1be out of bounds>, len = 16}, ttl = { s = 0x7fd6602bb550 "\224\245&`\326\177", len = 1613349696}, user_param = {s = 0x1 <Address 0x1 out of bounds>, len = 1}, maddr = {s = 0x7fffab3a3e8c "\026", len = 1613346624}, method = { s = 0x7fffab3a3ef0 "", len = -1422246496}, lr = { s = 0x7fd657c0ba58 "\020\363NX\326\177", len = 1613346624}, r2 = { s = 0x7fd657c0ba60 "(\363NX\326\177", len = 1470086695}, gr = { s = 0x7fd660248010 "", len = 5426506}, transport_val = { s = 0x7fd66029c308 "drecv\r\n", len = 1584265709}, ttl_val = { s = 0x7fd660248010 "", len = 5424982}, user_param_val = { s = 0xfbad8001 <Address 0xfbad8001 out of bounds>, len = 1584265709}, maddr_val = {s = 0x7fd660248010 "", len = 5424982}, method_val = {s = 0x1 <Address 0x1 out of bounds>, len = 1613349664}, lr_val = {s = 0x7fd66029c308 "drecv\r\n", len = 1472248408}, r2_val = {s = 0x1 <Address 0x1 out of bounds>, len = -1422246160}, gr_val = {s = 0x8c0d00 "\020\200$`\326\177", len = 1472248408}} u = <value optimized out> port = <value optimized out> i = 3 flags = <value optimized out> ---Type <return> to continue, or q <return> to quit--- avp = <value optimized out> st = {flags = 0, id = 0, name = {n = 0, s = {s = 0x0, len = 0}, re = 0x0}, avp = 0x0} sct = <value optimized out> sjt = <value optimized out> rve = <value optimized out> mct = <value optimized out> rv = 0x7fd6585f05e8 rv1 = <value optimized out> c1 = {cache_type = 2872721136, val_type = 32767, c = {avp_val = { n = 0, s = { s = 0x1600000000 <Address 0x1600000000 out of bounds>, len = 0}, re = 0x1600000000}, pval = {rs = { s = 0x1600000000 <Address 0x1600000000 out of bounds>, len = 0}, ri = 1613371344, flags = 32726}}, i2s = "\360>:\253\377\177\000\000 \276\300W\326\177\000\000\000\200\200\200\000"} s = {s = 0x0, len = 1482622128} srevp = {0x7fd66026b2a8, 0x7fd6602a17d0} mod_f_params = {{type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, ---Type <return> to continue, or q <return> to quit--- select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}, {type = NOSUBTYPE, u = {number = 0, string = 0x0, str = {s = 0x0, len = 0}, data = 0x0, attr = 0x0, select = 0x0}}} #10 0x000000000041e406 in run_actions (h=0x7fffab3a4050, a=<value optimized out>, msg=<value optimized out>) at action.c:1644 t = 0x7fd66026b2a8 ret = 1 mod = <value optimized out> ms = 1 #11 0x000000000041e9b0 in run_top_route (a=0x7fd66026ae28, msg=0x7fd6602a17d0, c=<value optimized out>) at action.c:1729 ctx = {rec_lev = 4, run_flags = 0, last_retcode = -1, jmp_env = {{ __jmpbuf = {140736066109520, -5014889658125657302, ---Type <return> to continue, or q <return> to quit--- 140558712876584, 140558713100240, 140736066110388, 140558582350304, 5015002385987394346, -5014889100588098774}, __mask_was_saved = 0, __saved_mask = {__val = {140558582351024, 140558582351059, 1482622170, 1331, 140558683994605, 140558712733712, 140558713100240, 140558713077664, 140558582351024, 140558582351068, 140558582352355, 9007199254740992, 5448801, 140558582351024, 140558677144600, 1073741825}}}}} p = 0x7fffab3a4050 ret = <value optimized out> sfbk = 0 #12 0x00000000004971de in receive_msg ( buf=0x7fd6585f04b0 "INVITE sip:00989989899@172.30.1.76 SIP/2.0\r\nVia: SIP/2.0/tls 172.30.1.241:56064;rport;branch=z9hG4bKPjnqn3DP0r7R7CrPY6VjzLVyXbiQMxrmkk\r\nMax-Forwards: 69\r\nFrom: "Samuel" sip:sam@172.30.1.76;tag=vasoN"..., len=<value optimized out>, rcv_info=0x7fd6585f01e0) at receive.c:209 msg = 0x7fd6602a17d0 ctx = {rec_lev = -1422245168, run_flags = 32767, last_retcode = 34, jmp_env = {{__jmpbuf = {0, 140558724835090, 140557099728897, 0, 140736065978768, 0, 65536, 140736066110032}, __mask_was_saved = -1422311024, __saved_mask = {__val = { 6021544150394, 65536, 10215065125, 3917010229, 140558582351024, 140558582350280, 4294967295, ---Type <return> to continue, or q <return> to quit--- 140736066110396, 140736066110388, 140558582350304, 5312548, 140736066110032, 5361791, 140736066110192, 140736066110160, 0}}}}} ret = <value optimized out> inb = { s = 0x7fd6585f04b0 "INVITE sip:00989989899@172.30.1.76SIP/2.0\r\nVia: SIP/2.0/tls 172.30.1.241:56064;rport;branch=z9hG4bKPjnqn3DP0r7R7CrPY6VjzLVyXbiQMxrmkk\r\nMax-Forwards: 69\r\nFrom: "Samuel" sip:sam@172.30.1.76;tag=vasoN"..., len = 1331} #13 0x000000000051115d in tcp_read_req (con=0x7fd6585f01c8, bytes_read=0x7fffab3a43bc, read_flags=0x7fffab3a43b4) at tcp_read.c:1231 bytes = <value optimized out> total_bytes = 1331 resp = <value optimized out> size = <value optimized out> dst = {send_sock = 0x14, to = {s = {sa_family = 1, sa_data = "\000\000\001\000\000\000\001 \000\000 \333+`"}, sin = {sin_family = 1, sin_port = 0, sin_addr = {s_addr = 1}, sin_zero = "\001 \000\000 \333+`"}, sin6 = {sin6_family = 1, sin6_port = 0, sin6_flowinfo = 1, sin6_addr = {__in6_u = { __u6_addr8 = "\001 \000\000 \333+`\326\177\000\000\000\000\000", __u6_addr16 = {8193, 0, 56096, 24619, 32726, 0, 0, 0}, __u6_addr32 = {8193, 1613486880, 32726, 0}}}, sin6_scope_id = 2872722344}}, ---Type <return> to continue, or q <return> to quit--- id = 32767, proto = 8 '\b', send_flags = {f = 0 '\000', blst_imask = 0 '\000'}} c = 99 'c' ret = <value optimized out> #14 0x0000000000512d73 in handle_io (fm=0x7fd6602bdb20, events=<value optimized out>, idx=-1) at tcp_read.c:1461 ret = <value optimized out> n = <value optimized out> read_flags = 1 con = 0x7fd6585f01c8 s = <value optimized out> resp = 140558713215776 t = <value optimized out> #15 0x0000000000514319 in io_wait_loop_epoll (unix_sock=<value optimized out>) at io_wait.h:1092 revents = 2 n = 1 fm = 0x7fd6602a764f #16 tcp_receive_loop (unix_sock=<value optimized out>) at tcp_read.c:1572 No locals. #17 0x00000000004e7c83 in tcp_init_children () at tcp_main.c:4952 r = 2 i = <value optimized out> ---Type <return> to continue, or q <return> to quit--- reader_fd_1 = 34 pid = 1970631271 si_desc = "tcp receiver (generic)\000\000\000\000\000\000\000\000\000\000\350J&`\326\177\000\000\225\070\177W\326\177\000\000\004\000\000\000\000\000\000\000\270\061+`\326\177\000\000T\346g\027\000\000\000\000\020\f\177W\326\177\000\000hM&`\326\177", '\000' <repeats 11 times>, "F:\253\377\177\000\000\026\331M\000\000\000\000\000`7Z", '\000' <repeats 12 times> si = <value optimized out> #18 0x0000000000464658 in main_loop () at main.c:1718 i = 4 pid = <value optimized out> si = 0x0 si_desc = "udp receiver child=3 sock=172.30.1.76:5060 \000W\326\177\000\000@ \202&`\326\177\000\000\005\000\000\000\t\000\000\000\030\301\206Z\326\177\000\000\001\000\000\000\000\000\000\000\300j\214\000\000\000\000\000\000\000@\002", '\000' <repeats 12 times>, "\005\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000W\J\000\000\000\000" nrprocs = 4 #19 0x000000000046607c in main (argc=<value optimized out>, argv=0x0) at main.c:2546 cfg_stream = <value optimized out> c = <value optimized out> r = 1970631271 ---Type <return> to continue, or q <return> to quit--- tmp = 0x41597b "H\203\304\b\303\377\065:\017?" tmp_len = 0 port = <value optimized out> proto = <value optimized out> ret = <value optimized out> seed = 737621369 rfd = <value optimized out> debug_save = <value optimized out> debug_flag = 37748736 dont_fork_cnt = 0 n_lst = 0x4 p = <value optimized out> (gdb)
So it is at different points in config, it works if you don't set flag 22, which is used for sip trace.
I see you are using tcp, can you try the latest version of git branch 3.3. There was a fix related to tcp after 3.3.1 and it is better to be sure it is not actually that case - a guide is available at:
http://www.kamailio.org/wiki/install/3.3.x/git
Obviously seems to be some memory corruption (potentially buffer overflow), because it is accessing a pointer which is not null, but invalid.
Cheers, Daniel
On 10/9/12 3:34 PM, samuel wrote:
As said in the previous mail, the problem is not always in the same point of the config. I just created a second crash: Oct 9 15:30:13 kamailio1 kamailio[18152]: INFO: <core> [mem/f_malloc.c:529]: freeing a free fragment (0x7fd66029c2f8/0x7fd66029c308) - ignore Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=520 a=6 n=route Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=667 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=632 a=25 n=has_totag Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=664 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=635 a=25 n=loose_route Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=651 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=646 a=26 n=is_method Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=17 n=if Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=651 a=26 n=is_method Oct 9 15:30:13 kamailio1 kamailio[18152]: ERROR: *** cfgtrace: c=[/usr/local/etc/kamailio/kamailio.cfg] l=663 a=27 n=sl_send_reply Oct 9 15:30:13 kamailio1 kernel: [6556831.473688] kamailio[18152] general protection ip:47b9d5 sp:7fffab3a22d0 error:0 in kamailio[400000+207000] Oct 9 15:30:13 kamailio1 kamailio[18155]: : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 30 Oct 9 15:30:13 kamailio1 kamailio[18137]: ALERT: <core> [main.c:785]: child process 18152 exited by a signal 11 Oct 9 15:30:13 kamailio1 kamailio[18137]: ALERT: <core> [main.c:788]: core was generated
last lines 6xx are the usual handling of request within dialog, being the last one a sl_send_reply.
Yesterday I could not test it but this morning just tested git version:
kamailio -V version: kamailio 3.3.1 (x86_64/linux) befbee flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: befbee compiled on 08:57:18 Oct 10 2012 with gcc 4.4.5
And the problem persists. I've tried to use TLS, TCP, and UDP transports from the User Agent and in all cases there is a problem when the setflag of siptrace module is set.
How can I debug further the issue?
Thanks a lot indeed for your support!!
Samuel.
Do you got a new backtrace, still happening at different point in config?
Can you try to compile with MEMDBG=1 and test again? Maybe we catch some faulty memop.
Cheers, Daniel
On 10/10/12 9:33 AM, samuel wrote:
Yesterday I could not test it but this morning just tested git version:
kamailio -V version: kamailio 3.3.1 (x86_64/linux) befbee flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: befbee compiled on 08:57:18 Oct 10 2012 with gcc 4.4.5
And the problem persists. I've tried to use TLS, TCP, and UDP transports from the User Agent and in all cases there is a problem when the setflag of siptrace module is set.
How can I debug further the issue?
Thanks a lot indeed for your support!!
Samuel.
Hi Samuel,
Could you try to test again with the updated siptrace sources from 3.3 branch? I had a patch recently commited for a crash in siptrace, so I'm pretty sure you have the same issue. Please tell us if your problem was fixed.:)
Dragos
On 10/09/2012 12:00 PM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
I'll try the last branch to see whether it solves the issue.
thanks a lot indeed for the fast attention and reaction!
Samuel.
On 16 October 2012 12:12, Dragos Dinu dragos.dinu@1and1.ro wrote:
Hi Samuel,
Could you try to test again with the updated siptrace sources from 3.3 branch? I had a patch recently commited for a crash in siptrace, so I'm pretty sure you have the same issue. Please tell us if your problem was fixed.:)
Dragos
On 10/09/2012 12:00 PM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
______________________________**_________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/**cgi-bin/mailman/listinfo/sr-**usershttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi all,
I had some problems with git (was it git fetch to update the local copy from the "main" reposiory?) and I manually changed siptrace.c with the addition of the len pointer in case it was using version 2 of HEP. So far the server has not crashed. Before with just a few calls (less than 10) the pointer got lost but now, with almost a hundred calls, kamailio is solid as a rock.
Thanks a lot, specially to Daniel and Dragos, for the attention received!
Samuel.
On 16 October 2012 13:02, samuel samu60@gmail.com wrote:
I'll try the last branch to see whether it solves the issue.
thanks a lot indeed for the fast attention and reaction!
Samuel.
On 16 October 2012 12:12, Dragos Dinu dragos.dinu@1and1.ro wrote:
Hi Samuel,
Could you try to test again with the updated siptrace sources from 3.3 branch? I had a patch recently commited for a crash in siptrace, so I'm pretty sure you have the same issue. Please tell us if your problem was fixed.:)
Dragos
On 10/09/2012 12:00 PM, samuel wrote:
Hi folks,
I was making some tests with kamailio (both 3.3.0 and 3.3.1 tgz) with siptrace module sending "homer" packets to a remote central agent where the data is stored and presented to the user.
The related config is: # ----- siptrace params from HOMER ----- modparam("siptrace", "duplicate_uri", "sip:A.B.C.D:9060") modparam("siptrace", "trace_to_database", 0) modparam("siptrace", "trace_flag", 22) modparam("siptrace", "trace_on", 1) #HEPv2 == timestamp will be included to HEP header modparam("siptrace", "hep_version", 2) modparam("siptrace", "hep_mode_on", 1) modparam("siptrace", "hep_capture_id", 1)
basically for all incoming messages, a route is called with the following actions:
route[HOMER] { setflag(22); sip_trace(); }
I've been making some tests and if I disable the setflag action, there is no problem (well, no transaction messages are sent to the central agent), but as soon as I activated setflag, kamailio crashes with the following error:
<core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 28
There's no concrete point in the config file where it fails so I think the problem is more tm-internals related. I can provide further information in case you need it.
Just for adding information, and without any intention to start a flame-war, with opensips 1.8.0-tls the server does not crash...
Thanks in advance! Samuel.
______________________________**_________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/**cgi-bin/mailman/listinfo/sr-**usershttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users