[sr-dev] [kamailio/kamailio] Kamailio resets $ru in branch_route context, probably race condition (#2708)

### Description Kamailio resets the $ru in a branch_route context, probably due a race condition and/or bug. Scenario: - Calls is established and answered with 404 - Kamailio enters failure_route - failure_route calls t_relay() - branch_route is entered - new branch is created with $ru = $ru; - next operation trying to access content of the $ru fails: Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: pv [pv_core.c:325]: pv_get_ruri_attr(): failed to parse the R-URI ### Troubleshooting Happens on a production system, after upgrade from 5.2.x to 5.3.9. Research regarding similar bugs have not produced something useful so far. #### Reproduction Issue happens less than 10 times a day and so far could not be easily reproduced. #### Debugging Data Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: NOTICE: 7e8bb2ba47f5ccc |-----> Failure Code 404<- Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[Failure] c=[/etc/kamailio/kamailio.cfg] l=1720 a=5 n=route Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=703 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=699 a=25 n=is_method Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=702 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=701 a=25 n=t_is_set Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=701 a=25 n=t_on_branch Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=707 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=703 a=25 n=is_method Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=706 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=705 a=25 n=t_is_set Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=707 a=25 n=t_on_branch_failure Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=720 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=711 a=25 n=is_method Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=714 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=713 a=25 n=t_is_set Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=713 a=25 n=t_on_failure Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=742 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:failure_route=[RELAY] c=[/etc/kamailio/kamailio.cfg] l=720 a=24 n=t_relay Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:branch_route=[MANAGE_BRANCH] c=[/etc/kamailio/kamailio.cfg] l=2142 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:branch_route=[MANAGE_BRANCH] c=[/etc/kamailio/kamailio.cfg] l=2135 a=63 n=assign Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:branch_route=[MANAGE_BRANCH] c=[/etc/kamailio/kamailio.cfg] l=2147 a=16 n=if Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:branch_route=[MANAGE_BRANCH] c=[/etc/kamailio/kamailio.cfg] l=2142 a=25 n=is_method Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: *** cfgtrace:branch_route=[MANAGE_BRANCH] c=[/etc/kamailio/kamailio.cfg] l=2142 a=63 n=assign Apr 13 11:44:39 proxy1 /usr/sbin/kamailio[2202]: ERROR: pv [pv_core.c:325]: pv_get_ruri_attr(): failed to parse the R-URI #### Log Messages See cfgtrace logging above. #### SIP Traffic Not easily to provide as its a productive system in a clustered setup. ### Possible Solutions Ideas on how to further debug this error are welcome. ### Additional Information * kamailio 5.3.9 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2708