Wrong dialog selected in on_reply/failure route in case of spirals.

List overview All Threads
Download

newer

older

CDRs and MongoDB

mistakenly detected NAT in IPv6...

Julia Boudniatsky

9 Dec 2014 9 Dec '14

8:06 p.m.

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150)| |<--------------------------------------------------------------| | | Thank you, Julia.

Attachments:

attachment.html (text/html — 9.3 KB)
dlg_cb_debug_3.txt (text/plain — 35.5 KB)

Show replies by date

Daniel-Constantin Mierla

10 Dec 10 Dec

12:21 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello, can you try with latest git branch 4.1 and report the results? I just pushed a patch that should fix it. Cheers, Daniel On 09/12/14 19:06, Julia Boudniatsky wrote:

...

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Julia Boudniatsky

2:39 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

...

Hello, can you try with latest git branch 4.1 and report the results? I just pushed a patch that should fix it. Cheers, Daniel On 09/12/14 19:06, Julia Boudniatsky wrote: After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

Daniel-Constantin Mierla

2:50 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello, do you see in logs the message printed by: LM_DBG("resetting the local dialog shortcuts\n"); Cheers, Daniel On 10/12/14 13:39, Julia Boudniatsky wrote:

...

Hello Daniel, I have tried from latest git branch 4.1 (I see your new patches for dialog ) But the problem steel occur. Thank you, Julia On Wed, Dec 10, 2014 at 12:21 PM, Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote: Hello, can you try with latest git branch 4.1 and report the results? I just pushed a patch that should fix it. Cheers, Daniel On 09/12/14 19:06, Julia Boudniatsky wrote:

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Daniel-Constantin Mierla

16 Dec 16 Dec

1:03 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

...

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Julia Boudniatsky

5:01 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello Daniel, Thank you for your reply. We set module dialog with: modparam("dialog", "detect_spirals", 1) and modparam("dialog", "dlg_match_mode", 1) I analyze log and found that "parse_to_param(): DEBUG: add_param: tag=" is missing in case of wrong dialog allocation correct dialog =========== dialog:: hash=1561:6164 state:: 1 ref_count:: 1 timestart:: 0 timeout:: 0 callid:: 8-12006(a)10.25.153.150 from_uri:: sip:039988222@10.25.153.150;user=phone from_tag:: 8 caller_contact:: sip:10.25.153.150:5062 caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 callee_bind_addr:: to_uri:: sip:D39539988200@10.25.153.150;user=phone to_tag:: callee_contact:: callee_cseq:: callee_route_set:: dialog detected in 301 ================ dialog:: hash=3838:6551 state:: 5 ref_count:: 1 timestart:: 1418731671 timeout:: 0 callid:: 7-12006(a)10.25.153.150 from_uri:: sip:039988222@10.25.153.150;user=phone from_tag:: 7 caller_contact:: sip:10.25.153.150:5062 caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 callee_bind_addr:: udp:10.25.153.150:5060 to_uri:: sip:D39539988200@10.25.153.150;user=phone to_tag:: 177 callee_contact:: sip:10.25.153.150:5061;transport=UDP callee_cseq:: 0 callee_route_set:: <sip:10.25.154.104;lr;ftag=7;vst=AAAAAHQAAAwADw8MAHIBAG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ-->,<sip:10.25.153.150;lr;ftag=7;vst=AAAAAHQAAAAADw8HDnYFCG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ--> Log debug3 receive1 INVITE =========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin//kamailio[5546]: DEBUG: dialog [dlg_hash.c:720]: internal_get_dlg(): no dialog callid=' 8-12006(a)10.25.153.150' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:751]: get_dlg(): no dialog callid='8-12006(a)10.25.153.150' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_handlers.c:1083]: dlg_get_msg_dialog(): dlg with callid ' 8-12006(a)10.25.153.150' not found dlg_manage(); ========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 send INVITE to Server 1 ================ receive from Server 1 (spirals) INVITE ================ different "To" user - case of number translation Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_profile.c:427]: set_current_dialog(): setting current dialog [1561:6164] send INVITE to Server 2 ================ "To" = "To" in INVITE to Server 1 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid=' 8-12006(a)10.25.153.150' found on entry 1561, dir=1 100 from Server 1 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 3 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 100 from Server 2 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header !!!!! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:176]: parse_to_param(): DEBUG: add_param: tag=8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:898]: parse_addr_spec(): end of header reached, state=29 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid=' 8-12006(a)10.25.153.150' found on entry 1561, dir=1 301 from Server 2 ============= Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [receive.c:152]: receive_msg(): After parse_msg... Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1072]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=644 T start=0xffffffffffffffff Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:949]: t_reply_matching(): DEBUG: t_reply_matching: hash 13904 label 0 branch 0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1004]: t_reply_matching(): DEBUG: t_reply_matching: reply matched (T=0x7fb318dcfae8)! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1141]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=645 T end=0x7fb318dcfae8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_reply.c:2206]: reply_received(): DEBUG: reply_received: org. status uas=100, uac[0]=100 local=0 is_invite=1) Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:204]: get_hdr_field(): DEBUG: get_hdr_body : content_length=0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6551 found on entry 3838 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d989c0 with 1 -> 2 BR, Julia On Tue, Dec 16, 2014 at 1:03 PM, Daniel-Constantin Mierla <miconda(a)gmail.com

...

wrote: Hello, looking again at your diagram, it seems that the Call-Id is the same, but there is a different dialog structure. If you dump the dialogs with kamctl, can you see the two structure listed? Are they with same dialog attributes (From/To/...)? Dialog module has a parameter for detecting spirals, have you set it to some value? Cheers, Daniel On 10/12/14 13:39, Julia Boudniatsky wrote: Hello Daniel, I have tried from latest git branch 4.1 (I see your new patches for dialog ) But the problem steel occur. Thank you, Julia On Wed, Dec 10, 2014 at 12:21 PM, Daniel-Constantin Mierla < miconda(a)gmail.com> wrote:

-- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Daniel-Constantin Mierla

8 Jan 8 Jan

2:06 a.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

...

Hello Daniel, Thank you for your reply. We set module dialog with: modparam("dialog", "detect_spirals", 1) and modparam("dialog", "dlg_match_mode", 1) I analyze log and found that "parse_to_param(): DEBUG: add_param: tag=" is missing in case of wrong dialog allocation correct dialog =========== dialog:: hash=1561:6164 state:: 1 ref_count:: 1 timestart:: 0 timeout:: 0 callid:: 8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150> from_uri:: sip:039988222@10.25.153.150 <mailto:sip%3A039988222@10.25.153.150>;user=phone from_tag:: 8 caller_contact:: sip:10.25.153.150:5062 <http://10.25.153.150:5062> caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> callee_bind_addr:: to_uri:: sip:D39539988200@10.25.153.150 <mailto:sip%3AD39539988200@10.25.153.150>;user=phone to_tag:: callee_contact:: callee_cseq:: callee_route_set:: dialog detected in 301 ================ dialog:: hash=3838:6551 state:: 5 ref_count:: 1 timestart:: 1418731671 timeout:: 0 callid:: 7-12006(a)10.25.153.150 <mailto:7-12006@10.25.153.150> from_uri:: sip:039988222@10.25.153.150 <mailto:sip%3A039988222@10.25.153.150>;user=phone from_tag:: 7 caller_contact:: sip:10.25.153.150:5062 <http://10.25.153.150:5062> caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> callee_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> to_uri:: sip:D39539988200@10.25.153.150 <mailto:sip%3AD39539988200@10.25.153.150>;user=phone to_tag:: 177 callee_contact:: sip:10.25.153.150:5061;transport=UDP callee_cseq:: 0 callee_route_set:: <sip:10.25.154.104;lr;ftag=7;vst=AAAAAHQAAAwADw8MAHIBAG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ-->,<sip:10.25.153.150;lr;ftag=7;vst=AAAAAHQAAAAADw8HDnYFCG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ--> Log debug3 receive1 INVITE =========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin//kamailio[5546]: DEBUG: dialog [dlg_hash.c:720]: internal_get_dlg(): no dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:751]: get_dlg(): no dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_handlers.c:1083]: dlg_get_msg_dialog(): dlg with callid '8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' not found dlg_manage(); ========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 send INVITE to Server 1 ================ receive from Server 1 (spirals) INVITE ================ different "To" user - case of number translation Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_profile.c:427]: set_current_dialog(): setting current dialog [1561:6164] send INVITE to Server 2 ================ "To" = "To" in INVITE to Server 1 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found on entry 1561, dir=1 100 from Server 1 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 3 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 100 from Server 2 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header !!!!! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:176]: parse_to_param(): DEBUG: add_param: tag=8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:898]: parse_addr_spec(): end of header reached, state=29 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found on entry 1561, dir=1 301 from Server 2 ============= Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [receive.c:152]: receive_msg(): After parse_msg... Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1072]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=644 T start=0xffffffffffffffff Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:949]: t_reply_matching(): DEBUG: t_reply_matching: hash 13904 label 0 branch 0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1004]: t_reply_matching(): DEBUG: t_reply_matching: reply matched (T=0x7fb318dcfae8)! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1141]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=645 T end=0x7fb318dcfae8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_reply.c:2206]: reply_received(): DEBUG: reply_received: org. status uas=100, uac[0]=100 local=0 is_invite=1) Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:204]: get_hdr_field(): DEBUG: get_hdr_body : content_length=0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6551 found on entry 3838 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d989c0 with 1 -> 2 BR, Julia On Tue, Dec 16, 2014 at 1:03 PM, Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote: Hello, looking again at your diagram, it seems that the Call-Id is the same, but there is a different dialog structure. If you dump the dialogs with kamctl, can you see the two structure listed? Are they with same dialog attributes (From/To/...)? Dialog module has a parameter for detecting spirals, have you set it to some value? Cheers, Daniel On 10/12/14 13:39, Julia Boudniatsky wrote:

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Julia Boudniatsky

14 Jan 14 Jan

3:43 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello Daniel, Please find in attached file the debug 3 log. Failed call with Call-ID: 4-21712(a)10.25.153.150. The problem occurred with all modes of "dlg_match_mode " The same load test in 3.3 executed without failures. Thank you, Julia. On Thu, Jan 8, 2015 at 2:06 AM, Daniel-Constantin Mierla <miconda(a)gmail.com> wrote:

...

Hello, can you send all the log messages with debug=3? You selected some in your previous post, but are not enough to get the entire picture of what messages, transactions and dialogs are processed. You can make a tgz and send it to me directly if the size is too big. Cheers, Daniel On 16/12/14 16:01, Julia Boudniatsky wrote: Hello Daniel, Thank you for your reply. We set module dialog with: modparam("dialog", "detect_spirals", 1) and modparam("dialog", "dlg_match_mode", 1) I analyze log and found that "parse_to_param(): DEBUG: add_param: tag=" is missing in case of wrong dialog allocation correct dialog =========== dialog:: hash=1561:6164 state:: 1 ref_count:: 1 timestart:: 0 timeout:: 0 callid:: 8-12006(a)10.25.153.150 from_uri:: sip:039988222@10.25.153.150;user=phone from_tag:: 8 caller_contact:: sip:10.25.153.150:5062 caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 callee_bind_addr:: to_uri:: sip:D39539988200@10.25.153.150;user=phone to_tag:: callee_contact:: callee_cseq:: callee_route_set:: dialog detected in 301 ================ dialog:: hash=3838:6551 state:: 5 ref_count:: 1 timestart:: 1418731671 timeout:: 0 callid:: 7-12006(a)10.25.153.150 from_uri:: sip:039988222@10.25.153.150;user=phone from_tag:: 7 caller_contact:: sip:10.25.153.150:5062 caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 callee_bind_addr:: udp:10.25.153.150:5060 to_uri:: sip:D39539988200@10.25.153.150;user=phone to_tag:: 177 callee_contact:: sip:10.25.153.150:5061;transport=UDP callee_cseq:: 0 callee_route_set:: <sip:10.25.154.104;lr;ftag=7;vst=AAAAAHQAAAwADw8MAHIBAG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ--> , <sip:10.25.153.150;lr;ftag=7;vst=AAAAAHQAAAAADw8HDnYFCG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ--> Log debug3 receive1 INVITE =========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin//kamailio[5546]: DEBUG: dialog [dlg_hash.c:720]: internal_get_dlg(): no dialog callid=' 8-12006(a)10.25.153.150' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:751]: get_dlg(): no dialog callid=' 8-12006(a)10.25.153.150' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_handlers.c:1083]: dlg_get_msg_dialog(): dlg with callid ' 8-12006(a)10.25.153.150' not found dlg_manage(); ========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 send INVITE to Server 1 ================ receive from Server 1 (spirals) INVITE ================ different "To" user - case of number translation Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_profile.c:427]: set_current_dialog(): setting current dialog [1561:6164] send INVITE to Server 2 ================ "To" = "To" in INVITE to Server 1 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid=' 8-12006(a)10.25.153.150' found on entry 1561, dir=1 100 from Server 1 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 3 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 100 from Server 2 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header !!!!! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:176]: parse_to_param(): DEBUG: add_param: tag=8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:898]: parse_addr_spec(): end of header reached, state=29 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid=' 8-12006(a)10.25.153.150' found on entry 1561, dir=1 301 from Server 2 ============= Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [receive.c:152]: receive_msg(): After parse_msg... Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1072]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=644 T start=0xffffffffffffffff Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:949]: t_reply_matching(): DEBUG: t_reply_matching: hash 13904 label 0 branch 0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1004]: t_reply_matching(): DEBUG: t_reply_matching: reply matched (T=0x7fb318dcfae8)! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1141]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=645 T end=0x7fb318dcfae8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_reply.c:2206]: reply_received(): DEBUG: reply_received: org. status uas=100, uac[0]=100 local=0 is_invite=1) Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:204]: get_hdr_field(): DEBUG: get_hdr_body : content_length=0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6551 found on entry 3838 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d989c0 with 1 -> 2 BR, Julia On Tue, Dec 16, 2014 at 1:03 PM, Daniel-Constantin Mierla < miconda(a)gmail.com> wrote:

Hello, looking again at your diagram, it seems that the Call-Id is the same, but there is a different dialog structure. If you dump the dialogs with kamctl, can you see the two structure listed? Are they with same dialog attributes (From/To/...)? Dialog module has a parameter for detecting spirals, have you set it to some value? Cheers, Daniel On 10/12/14 13:39, Julia Boudniatsky wrote: Hello Daniel, I have tried from latest git branch 4.1 (I see your new patches for dialog ) But the problem steel occur. Thank you, Julia On Wed, Dec 10, 2014 at 12:21 PM, Daniel-Constantin Mierla < miconda(a)gmail.com> wrote:

-- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Daniel-Constantin Mierla

4:07 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

...

Hello Daniel, Please find in attached file the debug 3 log. Failed call with Call-ID: 4-21712(a)10.25.153.150 <mailto:4-21712@10.25.153.150>. The problem occurred with all modes of "dlg_match_mode " The same load test in 3.3 executed without failures. Thank you, Julia. On Thu, Jan 8, 2015 at 2:06 AM, Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote: Hello, can you send all the log messages with debug=3? You selected some in your previous post, but are not enough to get the entire picture of what messages, transactions and dialogs are processed. You can make a tgz and send it to me directly if the size is too big. Cheers, Daniel On 16/12/14 16:01, Julia Boudniatsky wrote:

Hello Daniel, Thank you for your reply. We set module dialog with: modparam("dialog", "detect_spirals", 1) and modparam("dialog", "dlg_match_mode", 1) I analyze log and found that "parse_to_param(): DEBUG: add_param: tag=" is missing in case of wrong dialog allocation correct dialog =========== dialog:: hash=1561:6164 state:: 1 ref_count:: 1 timestart:: 0 timeout:: 0 callid:: 8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150> from_uri:: sip:039988222@10.25.153.150 <mailto:sip%3A039988222@10.25.153.150>;user=phone from_tag:: 8 caller_contact:: sip:10.25.153.150:5062 <http://10.25.153.150:5062> caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> callee_bind_addr:: to_uri:: sip:D39539988200@10.25.153.150 <mailto:sip%3AD39539988200@10.25.153.150>;user=phone to_tag:: callee_contact:: callee_cseq:: callee_route_set:: dialog detected in 301 ================ dialog:: hash=3838:6551 state:: 5 ref_count:: 1 timestart:: 1418731671 timeout:: 0 callid:: 7-12006(a)10.25.153.150 <mailto:7-12006@10.25.153.150> from_uri:: sip:039988222@10.25.153.150 <mailto:sip%3A039988222@10.25.153.150>;user=phone from_tag:: 7 caller_contact:: sip:10.25.153.150:5062 <http://10.25.153.150:5062> caller_cseq:: 1 caller_route_set:: caller_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> callee_bind_addr:: udp:10.25.153.150:5060 <http://10.25.153.150:5060> to_uri:: sip:D39539988200@10.25.153.150 <mailto:sip%3AD39539988200@10.25.153.150>;user=phone to_tag:: 177 callee_contact:: sip:10.25.153.150:5061;transport=UDP callee_cseq:: 0 callee_route_set:: <sip:10.25.154.104;lr;ftag=7;vst=AAAAAHQAAAwADw8MAHIBAG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ-->,<sip:10.25.153.150;lr;ftag=7;vst=AAAAAHQAAAAADw8HDnYFCG4DBQADAB0fBAYeCwAADUVIEAFYAlUfBgpuZQ--> Log debug3 receive1 INVITE =========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin//kamailio[5546]: DEBUG: dialog [dlg_hash.c:720]: internal_get_dlg(): no dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:751]: get_dlg(): no dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_handlers.c:1083]: dlg_get_msg_dialog(): dlg with callid '8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' not found dlg_manage(); ========== Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5546]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 send INVITE to Server 1 ================ receive from Server 1 (spirals) INVITE ================ different "To" user - case of number translation Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_profile.c:427]: set_current_dialog(): setting current dialog [1561:6164] send INVITE to Server 2 ================ "To" = "To" in INVITE to Server 1 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5543]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found on entry 1561, dir=1 100 from Server 1 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d34fc8 with 1 -> 3 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5541]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6164 found on entry 1561 100 from Server 2 ============ Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header !!!!! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:176]: parse_to_param(): DEBUG: add_param: tag=8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: <core> [parser/parse_addr_spec.c:898]: parse_addr_spec(): end of header reached, state=29 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:711]: internal_get_dlg(): ref dlg 0x7fb318d34fc8 with 1 -> 2 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5544]: DEBUG: dialog [dlg_hash.c:714]: internal_get_dlg(): dialog callid='8-12006(a)10.25.153.150 <mailto:8-12006@10.25.153.150>' found on entry 1561, dir=1 301 from Server 2 ============= Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [receive.c:152]: receive_msg(): After parse_msg... Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1072]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=644 T start=0xffffffffffffffff Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:949]: t_reply_matching(): DEBUG: t_reply_matching: hash 13904 label 0 branch 0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1004]: t_reply_matching(): DEBUG: t_reply_matching: reply matched (T=0x7fb318dcfae8)! Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_lookup.c:1141]: t_check_msg(): DEBUG: t_check_msg: msg id=645 global id=645 T end=0x7fb318dcfae8 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: tm [t_reply.c:2206]: reply_received(): DEBUG: reply_received: org. status uas=100, uac[0]=100 local=0 is_invite=1) Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:204]: get_hdr_field(): DEBUG: get_hdr_body : content_length=0 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: <core> [parser/msg_parser.c:106]: get_hdr_field(): found end of header Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:656]: dlg_lookup(): dialog id=6551 found on entry 3838 Dec 16 14:07:49 KAMnet-JAF-TST-01 /usr/local/sbin/kamailio[5540]: DEBUG: dialog [dlg_hash.c:654]: dlg_lookup(): ref dlg 0x7fb318d989c0 with 1 -> 2 BR, Julia On Tue, Dec 16, 2014 at 1:03 PM, Daniel-Constantin Mierla <miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote: Hello, looking again at your diagram, it seems that the Call-Id is the same, but there is a different dialog structure. If you dump the dialogs with kamctl, can you see the two structure listed? Are they with same dialog attributes (From/To/...)? Dialog module has a parameter for detecting spirals, have you set it to some value? Cheers, Daniel On 10/12/14 13:39, Julia Boudniatsky wrote:

After kamailio upgrade from v3.3 to v4.1.7 in production ~5% of calls with spirals dialog are failed with wrong dialog variables in on-reply/failure routes. The problem has been reproduced in the LAB (load test generated by SIPP). Please see the attached file with debug3 log. Scenario: Proxy Location Kamailio P1 Nbr translation Redirect | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |<-------------------------------| | | 100 Trying | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | | |------------------------------->| | | | | | INVITE | | | dlg 0x7f8f8501aba0 | | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>) | |-------------------------------------------------------------->| | 100 Trying | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | 301 Moved Permanently | | dlg 0x7f8f8504cda0 !!!!!!!! | | Call-ID:(5-8286@10.25.153.150 <mailto:5-8286@10.25.153.150>)| |<--------------------------------------------------------------| | | Thank you, Julia. _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Daniel-Constantin Mierla

16 Jan 16 Jan

12:42 a.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello, I looked at the logs, but I cannot see the log message I added with the patch for reseting local dialog shortcuts after onreply and failure route. It shuld be something like: "... resetting the local dialog shortcuts" Are you sure you are running the latest branch 4.1? You can send the output of kamailio -v to double check here. I separated the logs per process and looked when a 301 was received, apparently the ones for Call-ID: 4-21712 are matching the same dialog. I found something not printing the same kind of logs like for other 301 for Call-ID: 2-21712. If you can point the line in the logs where you think it was the wrong dialog match, I can investigate from there, but now I couldn't find any lead. Anyhow, if it is the latest branch 4.1 and the problem still persists, can you add the following xlog at the beginning of main request_route (or the route {...} ), plus in each reply_route, onreply_route and failure_route: xlog("processing $rm ( $ci / %ft / $tt / $hdr(Via) ) [$fu => $tu]\n"); Then redo the tests to catch the issue and send me the new logs. Because you use sipp with tags like '4', it is hard to search on a single tocken and I want to have the all message attribute when a relevant route block is executed. Cheers, Daniel On 14/01/15 15:07, Daniel-Constantin Mierla wrote:

...

Hello Julia, ok, I will look over the logs very soon. As a short note for the future, make the archives zip or tgz, being more common formats that can be unarchived everywhere without installing additional app, being handy to open even on small devices. Now it is ok with rar, being at the desk. Cheers, Daniel On 14/01/15 14:43, Julia Boudniatsky wrote: > Hello Daniel, > > Please find in attached file the debug 3 log. > Failed call with Call-ID: 4-21712(a)10.25.153.150 > <mailto:4-21712@10.25.153.150>. > The problem occurred with all modes of "dlg_match_mode " > The same load test in 3.3 executed without failures. > > Thank you, > > Julia.

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Daniel-Constantin Mierla

25 Jan 25 Jan

12:03 a.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

...

-- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

Julia Boudniatsky

12:42 p.m.

New subject: Wrong dialog selected in on_reply/failure route in case of spirals.

Hello Daniel, I have re-installed kamailio from last 4.1 branch, but unfortunately, the problem persists. Failed call with callID=2-17858(a)10.25153.150. The log file i send to your private mail. Best regards, Julia On Sun, Jan 25, 2015 at 12:03 AM, Daniel-Constantin Mierla < miconda(a)gmail.com> wrote:

...

Hello, got the time to analyze your last logs and I discovered that the previous patch was not resetting properly the dialog shortcuts. I pushed a fix, can you try again with the latest version from one of the branches: master, 4.2 or 4.1?Let me know the results. Cheers, Daniel On 15/01/15 23:42, Daniel-Constantin Mierla wrote: Hello, I looked at the logs, but I cannot see the log message I added with the patch for reseting local dialog shortcuts after onreply and failure route. It shuld be something like: "... resetting the local dialog shortcuts" Are you sure you are running the latest branch 4.1? You can send the output of kamailio -v to double check here. I separated the logs per process and looked when a 301 was received, apparently the ones for Call-ID: 4-21712 are matching the same dialog. I found something not printing the same kind of logs like for other 301 for Call-ID: 2-21712. If you can point the line in the logs where you think it was the wrong dialog match, I can investigate from there, but now I couldn't find any lead. Anyhow, if it is the latest branch 4.1 and the problem still persists, can you add the following xlog at the beginning of main request_route (or the route {...} ), plus in each reply_route, onreply_route and failure_route: xlog("processing $rm ( $ci / %ft / $tt / $hdr(Via) ) [$fu => $tu]\n"); Then redo the tests to catch the issue and send me the new logs. Because you use sipp with tags like '4', it is hard to search on a single tocken and I want to have the all message attribute when a relevant route block is executed. Cheers, Daniel On 14/01/15 15:07, Daniel-Constantin Mierla wrote: Hello Julia, ok, I will look over the logs very soon. As a short note for the future, make the archives zip or tgz, being more common formats that can be unarchived everywhere without installing additional app, being handy to open even on small devices. Now it is ok with rar, being at the desk. Cheers, Daniel On 14/01/15 14:43, Julia Boudniatsky wrote: Hello Daniel, Please find in attached file the debug 3 log. Failed call with Call-ID: 4-21712(a)10.25.153.150. The problem occurred with all modes of "dlg_match_mode " The same load test in 3.3 executed without failures. Thank you, Julia. -- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda -- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda

3589

days inactive

3636

days old

sr-users@lists.kamailio.org

Manage subscription

11 comments

2 participants

tags (0)

participants (2)

Daniel-Constantin Mierla
Julia Boudniatsky