Thanks for the response Daniel,
I was trying to force the socket from the tm:local-request and this was causing me the crash. About this crash I wasn't aware about the ulimit -c.... I will change that in my servers!
And about mtree, do you have some explanation on how to use that for prefix routing purposes? Before choosing LCR as the prefix routing module for my solution I saw you mentioning to use it, but I could not find any explanation on how to use it for this purpose... So I chose LCR... It is complex but is Ok, however (taking the chance for a feedback) I do feel kamailio has a big problem with prefix routing.... all solutions/modules seem clumsy to manage and there isn't a no-brainer choice....

Thanks,
Patrick Wakano


On 22 September 2017 at 17:18, Daniel-Constantin Mierla <miconda@gmail.com> wrote:

Hello,


On 22.09.17 04:42, Patrick Wakano wrote:
Hello list,

After deep investigation I found out that the issue is happening because of the mechanism Kamailio uses to find the most appropriate socket combined with the virtual IP usage....
For socket selection, Kamailio first asks Linux which would be the routing for a certain destination, then matches this outcome with the list of listened sockets.
By default Linux will always return the local IP instead of the virtual one. So if Kamailio only binds to the virtual IP, the Linux's returned IP won't match our socket list and then the socket selection fails....
So currently Kamailio active/stand-by HA setup with virtual IP in a multi-homed environment does not work properly...
We can overcome this by listening to all addresses, but then Kamailio will decide to use the local IP instead of the virtual one, and then the HA idea end up broken....
We can also force socket for everything. For regular call routing, this options actually works fine if the script always force some socket, but it seems to be not always possible for local generated messages.

do you have an event_route[tm:local-request] in your config? There you should be able to force send socket for local requests, iirc.

By disabling mhomed and using the dispatcher socket attr to send the OPTIONS message it works for example. But for the LCR module we have no option because apparently the LCR module has only one socket option, and so I am not able have a different socket per gateway.
We can also explicitly set the src address for a routing rule to be the virtual IP instead of the local one. This makes Linux to return the virtual IP instead of the local one. This solution seems fine, but since it is done at linux level, every other service on the box would be affected and every route on the box would needed to be changed..
Another problem I found out is that I am not able to force the socket for local generated messages. It actually crashes Kamailio, and I always get a "core was not generated" msg.... anyone aware of that?

Have you done 'ulimit -c unlimited' and then run kamailio as root?

The actual fix for this, would be having Kamailio to search the most appropriate socket in a different manner. It could loop the sockets and see which one reaches the destination instead of asking Linux to do it, or it could have some routing directives which could be consulted before selecting a socket.... (just ideas, not sure how feasible they are...)

Should anyone make a PR for such a change, it will be obviously considered for merging.

I am considering using the Linux routing change option, but has anyone faced such situation before?

I typically do lcr by combining mtree+dispatcher, never run in such issue myself, but I also do mainly force send socket explicitly because makes it more clear in config.

Cheers,
Daniel



Regards,
Patrick Wakano


On 24 August 2017 at 09:18, Patrick Wakano <pwakano@gmail.com> wrote:
Hello Daniel,
I still couldn't make this SIP probing work.... So currently my setup cannot actively check for GW availability...
Any idea about what may be causing it?

Thanks for your time,
Patrick Wakano

On 16 August 2017 at 13:01, Patrick Wakano <pwakano@gmail.com> wrote:
Forgot to mention it is a centos7 box...

On 16 Aug. 2017 11:02 am, "Patrick Wakano" <pwakano@gmail.com> wrote:
Hi Daniel!
Very grateful for your attention!

Apologies for lacking important info, check below the version and sockets.
It may be worth mentioning these IPs are virtual ones and I am using net.ipv4.ip_nonlocal_bind = 1(in this case the virtual IP is assigned to the box)
Interesting that it has a "mhomed: no" for all sockets but I do have mhomed=1 in my cfg file.... could that be the issue?
Also there is no message before this one: /usr/sbin/kamailio[6164]: ERROR: <core> [core/forward.c:181]: get_out_socket(): no socket found
This is the first one that pops when dispatcher tries to ping....

Thanks,
Patrick

kamailio -v
version: kamailio 5.0.2 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled on 12:03:49 Jun 26 2017 with gcc 4.8.5

kamcmd> core.sockets_list
{
    socket: {
        proto: udp
        address: 172.28.128.100
        ipaddress: 172.28.128.100
        port: 5060
        mcast: no
        mhomed: no
    }
    socket: {
        proto: udp
        address: 192.168.33.100
        ipaddress: 192.168.33.100
        port: 5060
        mcast: no
        mhomed: no
    }
    socket: {
        proto: tcp
        address: 172.28.128.100
        ipaddress: 172.28.128.100
        port: 5060
        mcast: no
        mhomed: no
    }
    socket: {
        proto: tcp
        address: 192.168.33.100
        ipaddress: 192.168.33.100
        port: 5060
        mcast: no
        mhomed: no
    }
    socket: {
        proto: tcp
        address: 172.28.128.100
        ipaddress: 172.28.128.100
        port: 8080
        mcast: no
        mhomed: no
    }
}


On 15 August 2017 at 17:24, Daniel-Constantin Mierla <miconda@gmail.com> wrote:

Hello,

what are the sockets kamailio is listening on? See `kamcmd help` for the command that lists the sockets.

Also, what is the log messages before the first one you pasted, there should be another one printed from get_out_socket().

It is always important to provide kamailio version (kamailio -v) and the operating system you are using.

Cheers,
Daniel


On 14.08.17 14:41, Patrick Wakano wrote:
Hello Kamailio list,

Hope you all doing well!

I am stuck with one problem that should be incredibly easy to be done, but apparently it is not.... I want my dispatcher, LCR and nathelper modules to always SIP ping my SIP clients. I have a Kamailio based SBC that is separating two networks, and so I am using the 'mhomed' flag, but none of these SIP pings are working.... for now I will only ask about the dispatcher....
My dispatcher module has to talk with SIP servers in both networks, the thing is when I enable SIP ping for it (modparam("dispatcher", "ds_probing_mode", 1)), it gives me this type of error:
/usr/sbin/kamailio[6164]: ERROR: <core> [core/forward.c:181]: get_out_socket(): no socket found
/usr/sbin/kamailio[6164]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:192.168.33.110:6060)
/usr/sbin/kamailio[6164]: ERROR: tm [ut.h:317]: uri2dst2(): no corresponding socket found for "192.168.33.110" af 2 (udp:192.168.33.110:6060)
/usr/sbin/kamailio[6164]: ERROR: tm [uac.c:443]: t_uac_prepare(): no socket found
/usr/sbin/kamailio[6164]: ERROR: dispatcher [dispatch.c:2652]: ds_ping_set(): unable to ping [sip:192.168.33.110:6060]

After research it seems I need to manually set a "socket" attribute for each of my GWs in dispatcher list. After putting the socket parameter for the GW, I got another error:
/usr/sbin/kamailio[14190]: ERROR: <core> [core/socket_info.c:2046]: parse_protohostport(): bad port number in udp:192.168.33.100:5060
/usr/sbin/kamailio[14190]: ERROR: <core> [core/forward.c:181]: get_out_socket(): no socket found
/usr/sbin/kamailio[14190]: ERROR: <core> [core/forward.c:183]: get_out_socket(): no corresponding socket found for(udp:192.168.33.110:6060)
/usr/sbin/kamailio[14190]: ERROR: tm [ut.h:317]: uri2dst2(): no corresponding socket found for "192.168.33.110" af 2 (udp:192.168.33.110:6060)
/usr/sbin/kamailio[14190]: ERROR: tm [uac.c:443]: t_uac_prepare(): no socket found
/usr/sbin/kamailio[14190]: ERROR: dispatcher [dispatch.c:2652]: ds_ping_set(): unable to ping [sip:192.168.33.110:6060]


What can be wrong in my port info in the socket config? (I tried the socket with and without quotes but had no luck...)
select * from dispatcher where id=2;
 id | setid |       destination       | flags | priority |                    attrs                    |     description    
----+-------+-------------------------+-------+----------+---------------------------------------------+---------------------
  2 |     2 | sip:192.168.33.110:6060 |     0 |        0 | socket="udp:192.168.33.100:5060";rweight=50 |
(1 row)

Anyhow, I was expecting the 'mhomed' parameter to do the socket selection for me. Regular call routing either by the dispatcher or by LCR, also seems to require me to explicitly set the socket otherwise Kamailio (or the OS) may end up using the wrong socket. So, another question is why 'mhomed' does not dictate the whole socket selection Kamailio has to do regardless the module it is using to send SIP requests?

Sorry for the long e-mail....
Best regards,
Patrick Wakano


_______________________________________________
Kamailio (SER) - Development Mailing List
sr-dev@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-dev

-- 
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com




-- 
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com