### Description
I migrate one of our old project on native to KEMI on app_python3 and get strange behavior - after some time kamailio stops processing requests/replies wit message "ksr_request_route/ksr_reply_route not found or is not callable"
### Troubleshooting
After some research with debug=3 and reading debug logs i found, that some of routes i forgot to rename in calls t_on_reply() and t_on_failure() - because they generates "not found or is not callable" in logs too. Also i check all event routes and found that some event_callbacks point to wrong KEMI functions.
Events of the form (which gave me the idea that it was the wrong names) DEBUG: 887937087 INVITE app_python3 [python_exec.c:93]: apy_exec(): ksr_onrl_MANAGE not found or is not callable DEBUG: app_python3 [python_exec.c:93]: apy_exec(): ksr_dialog_event not found or is not callable presents in logs very rarely (I saw them only after hours of debugging) even after restart.
For me its look like that nonexisted routes completly stops interpretator and block all subsequent processing in the fork and kamailio very rarely has time to log it.
When i use right names of functions it works well and no "ksr_request_route/ksr_reply_route not found or is not callable"
PS: KSR.tm.t_on_branch() with wrong route generates normal exception and nonblock execution #### Reproduction
Just use wrong names in KSR.tm.t_on_reply()/KSR.tm.t_on_failure() or modparam event_callback in any module (i got it with dispatcher and dialog)
#### Log Messages Exception that generates by wrong function in KSR.tm.t_on_branch() ERROR: 63586697 BYE app_python3 [python_support.c:156]: python_handle_exception(): apy_exec: ksr_request_route((null)): Unhandled exception in the Python code: AttributeError: 'kamailio' object has no attribute 'ksr_br_MANAGE'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/etc/kamailio@REGION/python/request_route.py", line 38, in ksr_request_route if self.ksr_WITHINDLG() == -255: File "/etc/kamailio@REGION/python/ksr_WITHINDLG.py", line 27, in ksr_WITHINDLG return self.ksr_RELAY() File "/etc/kamailio@REGION/python/ksr_RELAY.py", line 17, in ksr_RELAY if KSR.tm.t_relay() < 0: SystemError: <built-in function t_relay> returned a result with an error set
### Possible Solutions
Use rigth names
### Additional Information
version: kamailio 5.7.1 (x86_64/linux) 4238e4 flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: 4238e4 compiled on 00:00:00 Sep 13 2022 with gcc 11.3.1
version: kamailio 5.6.4 (x86_64/linux) flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled with gcc 10.2.1
* **Operating System**:
PRETTY_NAME="Oracle Linux Server 9.2" PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
Can you try with git master branch? I pushed a commit trying to clear python error state in a few more places.
I am at conference in other city, on next week i will return on work and test patch
I build kamailio from master ``` $ /usr/local/sbin/kamailio -v version: kamailio 5.8.0-dev1 (x86_64/linux) 27f87c ```
and reproduce problem: ``` modparam("dispatcher", "event_callback", "ksr_evnt_dispatcher") # route actually named ksr_event_dispatcher KSR.tm.t_on_reply('ksr_onreply_MANAGE') # route not existed. KSR.tm.t_on_failure('ksr_failure_ROUTE') # route actually named ksr_failure_MANAGE ```
Patch work great - i get messages like ``` 0bdb87a767b52b60-1028569@10.100.32.228 OPTIONS app_python3 [python_exec.c:100]: apy_exec(): ksr_evnt_dispatcher not found or is not callable ((nil)) DEBUG: 766520133 INVITE app_python3 [python_exec.c:100]: apy_exec(): ksr_onreply_MANAGE not found or is not callable ((nil)) DEBUG: 2039759053 INVITE app_python3 [python_exec.c:100]: apy_exec(): ksr_failure_ROUTE not found or is not callable ((nil)) ```
and all forks still availible. But can those messages be at log level 'ERROR'? Without log level 'DEBUG' it's still hard to detect non-existed KEMI route
This issue is stale because it has been open 6 weeks with no activity. Remove stale label or comment or this will be closed in 2 weeks.
I pushed a patch to print error log messages when not finding the callback function for tm events (branch, failure, onreply). Not for event route callbacks because many of them are optional, to be executed if they exist. I will try to think if any option could be added to be able to control it (e.g., maybe a modparam).
Closing this one, originally reported issue fixed. If anyone wants to contribute further enhancements, PRs can be made.
Closed #3572 as completed.