OK, did a few more tests but have come accross something, which I am not sure is intended behaviour.
When setting a destingation as probing in failure route (due to timeout), the destination still gets used in destination selection.
# ./kamailio -V
version: kamailio 3.3.0-dev0 (i386/linux) 25bedc
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 25bedc
compiled on 09:18:41 Oct 21 2011 with gcc 4.1.2
Dispatcher module parameters (in testing) are as follows (SBC_PING_FROM is #defined previously):
modparam("dispatcher", "flags", 2)
modparam("dispatcher", "dst_avp", "$avp(AVP_DST)")
modparam("dispatcher", "grp_avp", "$avp(AVP_GRP)")
modparam("dispatcher", "cnt_avp", "$avp(AVP_CNT)")
modparam("dispatcher", "ds_ping_method", "OPTIONS")
modparam("dispatcher", "ds_ping_from", SBC_PING_FROM)
modparam("dispatcher", "ds_ping_interval", 10)
modparam("dispatcher", "ds_probing_threshhold", 1)
modparam("dispatcher", "ds_probing_mode", 0)
Main routing logic has following snippet to select destination (hash table selects dispatcher setid based on request domain):
if(!ds_select_dst("$sht(which_sbc=>$rd)", "0")) {
sl_send_reply("500", "No destination available");
xlog("route[MAIN] : $rm : No destinations available for $rd");
exit;
}
Failure route has following logic to select next destination based on timout/failure of destination:
if (t_branch_timeout() && !t_branch_replied())
{
xlog("route[TO_SBC] : $rm : timeout and no reply ($si:$sp->$Ri:$Rp->$du)\n");
xlog("route[TO_SBC] : $rm : setting $du to probing state");
ds_mark_dst("p");
if(ds_next_dst())
{
xlog("route[TO_SBC] : $rm : next destination select ($du)\n");
t_on_failure("TO_SBC");
t_relay();
exit;
} else {
send_reply("500", "No destination available");
xlog("route[TO_SBC] : $rm : No destinations available for $rd");
exit;
}
}
According to 3.2 module docs for dispatcher, when a destination is set into probing state, it will not be used by ds_select_dst:
----
4.6.
ds_mark_dst("s")Mark the last used address from destination set as inactive ("i"/"I"/"0"), active ("a"/"A"/"1") or probing ("p"/"P"/"2"). With this function, an automatic detection of failed gateways can be implemented. When an address is marked as inactive or probing, it will be ignored by 'ds_select_dst' and 'ds_select_domain'.
possible parameters:
"i", "I" or "0" - the last destination should be set to inactive and will be ignored in future requests.
"a", "A" or "1" - the last destination should be set to active and the error-counter should set to "0".
"p", "P" or "2" - the last destination will be set to probing. Note: You will need to call this function "threshhold"-times, before it will be actually set to probing.
This function can be used from REQUEST_ROUTE, FAILURE_ROUTE.
---
What happens here, for me, is:
[1] Gateway is in Active mode (state: AX).
[2] Request comes in and times out, destination is set to Active/Probing (state: AP)
[3] Another request comes in and it selects gateway that is in AP mode, times out, and then selects next dst in list.
[4] Another request comes in and it selects gateway that is in AP mode, times out, and then selects next dst in list.
.
.
NOTE: The requests selecting the AP mode gateway may not be right after each other (algorythm used is hash over callid) but I have stripped those out in above steps. If I'm not mistaked, if 2 destination in a set, and 1 destination is marked as AP, then remaining destination should always be selected as destination to send to. The destination marked AP (active-probing) should not be selected while in probing state.
When the gateway is set into AP mode at step [2], then, according to docs, any new request coming in should not have the gateway selected as it is marked as being in probing state.
Is this the intended behaviour or am I missing something in the documentation, or is it a bug?
Thanks
On 22/10/2011 15:55, Daniel-Constantin Mierla wrote:Hello,
yes, I will do it. Just happens to travel these days, but when I get the time for it, I will do it.
Btw, did you test also the load balancing functionality? Was it affected or all is ok when you change the states? I mean when you disable/make inactive a gateway, is no longer used for routing, right? Just double checking, to be sure it was not affected by this change.
Thanks,
Daniel
On 10/22/11 3:42 PM, Asgaroth wrote:
Hi Daniel,
Just a reminder for this issue, to backport to 3.2.0 :)
Thanks
On 21/10/2011 10:53, Asgaroth wrote:
Hi Daniel,
It appears the change you made has fixed the issue. Below are my tests:
# kamailio -V
version: kamailio 3.3.0-dev0 (i386/linux) 25bedc
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 25bedc
compiled on 09:18:41 Oct 21 2011 with gcc 4.1.2
Here we have the dispatcher state as loaded from db on startup in
kamailio memory.
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state for destination to inactive (Worked)
# kamctl fifo ds_set_state i 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state for destination to active (Worked)
# kamctl fifo ds_set_state a 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state for destination to inactive-probing (Worked)
# kamctl fifo ds_set_state ip 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
I waited for OPTIONS/INFO message and checked state (Worked), changed
state from IP -> AX
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state to active-probing (Worked)
# kamctl fifo ds_set_state ap 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
I waited for OPTIONS/INFO message and checked state (Worked), changed
state from AP -> AX
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state to disabled (Worked). At this point no pinging occured
as destination is admin down.
# kamctl fifo ds_set_state d 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=DX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now try set state to inactive-probing again, from disabled state (Worked)
# kamctl fifo ds_set_state ip 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
I waited for OPTIONS/INFO message and checked state (Worked), changed
state from IP -> AX
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Thanks.
On 20/10/2011 22:54, Daniel-Constantin Mierla wrote:
Hello,_______________________________________________
indeed the setting of active state via MI command was wrong. Should be
fixed now in master branch. Can you test it and see if works ok now (I
had no option to test it myself yet). If all goes fine with your
tests, then I will backport.
Guidelines to install master branch can be found at:
https://www.kamailio.org/wiki/install/devel/git
Cheers,
Daniel
On 10/20/11 7:52 PM, Asgaroth wrote:
Hi All,
Just been doing some testing with Kamailio 3.2 and the dispatcher
module, and have some strange behaviour, can we confirm if it is
expected behaviour or a bug of some sort.
It appears that I can only set the dspatcher state via fifo command
once, and not reset it again, here is an example:
version: kamailio 3.2.0 (i386/linux) 94d3b8
flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS,
DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC,
DBG_QM_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 94d3b8
compiled on 12:34:43 Oct 20 2011 with gcc 4.1.2
Here we start with the dispatcher list as it was pulled from the
database and currently loaded in memory.
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now I try to set one gateway inactive:
# kamctl fifo ds_set_state i 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
All normal so far, now if I try to set the same gateway active again,
the state still stays inactive, i dont seem to be able to change the
state back to active. The same thing happens if I attempt to set this
state to disabled and then try to set it back to active afterwards. The
state does not change from what the first attempt was.
# kamctl fifo ds_set_state a 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now I can try to set the state to inactive probing, that works.
# kamctl fifo ds_set_state ip 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Now if I try to disable the same destination:
# kamctl fifo ds_set_state d 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
State still stays in Inactive/Probing mode.
Now if I try to set the same gateway to active/probing mode, I cant:
# kamctl fifo ds_set_state ap 1 sip:1.1.1.1:10001
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=IP priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
The only way for me to get the state back to active after the above is
to do a reload on the dispatch table:
# kamctl dispatcher reload
# kamctl dispatcher dump
SET_NO:: 2
SET:: 1
URI:: sip:1.1.1.1:10001 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10000 flags=AX priority=0 attrs=
SET:: 2
URI:: sip:1.1.1.1:10003 flags=AX priority=0 attrs=
URI:: sip:1.1.1.1:10002 flags=AX priority=0 attrs=
Is the dispatcher expected to behave as above?
I'm still trying to understand the different states assuming the
following options are set in the configuration file:
modparam("dispatcher", "ds_ping_interval", 10)
modparam("dispatcher", "ds_probing_threshhold", 1)
modparam("dispatcher", "ds_probing_mode", 0)
AX = Active/No Probing
(*) Gateway is used in ds_select_*, ds_next_*
(*) No OPTIONS/INFO messages are sent to destination to check alive
state
This appears to be the normal operating state with no issues
detected.
AP = Active/Probing
(*) Gateway is NOT used in ds_select_*, ds_next_*
(*) OPTIONS/INFO messages ARE sent to destination to check alive
state
This appears to be the state taken when a gateway times out and
ds_mark_dst("p") is set in failure route, the gateway is "pinged" while
down and set into AX mode once a response is recieved.
IX = Inactive/No Probing
(*) Gateway is NOT used in ds_select_*, ds_next_*
(*) No OPTIONS/INFO messages sent to destination
This appears to be the state taken when a gateway times out and if
ds_mark_dst("i") is set in failure route, this gateway is set inactive
and no "pinging" is performed.
IP = Inactive/Probing
(*) Gateway is NOT used in ds_select_*, ds_next_*
(*) OPTIONS/INFO messages ARE sent to destination
I'm not sure when this state would be reached unless it is set by
fifo command. The ds_mark_dst function only allows "a","i" or "p". So
the probing method would only become active is ds_probing_mode = 1. In
my case it is 0, so I'm not sure if I would ever reach this state unless
I forced it. What would happen here is the gateway came back alive once
in "IP" state, would it go back to "AX" state?
DX = Disabled/No Probing
(*) Gateway is administratively set to down
This is where manual intervention has occured and the destination
has been set to administrativly down, for example, work being carried
out on the destination gateway.
All these states make sense, for the most part, I'm a little unsure of
how IP/IX is meant to work unless you manually set it to "inactive" in
your failure route.
The main concern though is the the fifo commands dont appear to be
working as expected, I can only set the state for a destination manually
once, and not again after that, unless I perform a dispatcher reload.
Has anyone else experienced this behaviour?
Any comments/suggestions explanations are most welcome.
Thanks
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
_______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users