[kamailio/kamailio] v5.0.4: nathelper: Errors sending ping messages when no default route set on host. (#1297)
### Description
When the nathelper module attempts to send OPTIONS ping messages on a host that has no default route set, the module errors out with one of two error messages.
``` Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: <core> [core/udp_server.c:591]: udp_send(): raw_iphdr_udp4_send(12,0x7f5be79587e6,4,...,212.2.172.228:39808,1500): Network is unreachable(101) Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: nathelper [nathelper.c:2077]: nh_timer(): udp_send failed ``` or, when I enable udpping_from_path the error message is as follows
``` Nov 1 18:57:29 localhost kam_registrar_1[2793]: ERROR: nathelper [nathelper.c:2073]: nh_timer(): send_raw from path failed Nov 1 18:57:49 localhost kam_registrar_1[2793]: ERROR: nathelper [nathelper.c:2073]: nh_timer(): send_raw from path failed ```
I am sending nat pings from a registrar behind a proxy. The registrar is on an internal network, the proxy has an interface on the internal network too.
I am storing the path information in location so that it can be used for the nat pings. The path information has the local internal address and port of the proxy to send the keepalive message via.
I have the "force_socket" parameter option set, which, I assume, is the socket to be used for all nat ping messages. This does not work as expected, I will open a seperate issue for this particular case.
I would have expected that the messages should be able to send to the proxy weather I have a default route set or not. NOTE: The default route, in this instance, is not on the same network that the sip traffic is flowing on.
I guess my expectation here is that if the destination to be sent to is on the same network as a socket that kamilio is listening on, then we should be able to send. In this case, the path is going to contain where to send to, but the actual contact of the aor is totally external.
### Troubleshooting
Module settings are as follows: registrar ``` modparam("registrar", "method_filtering", 1) modparam("registrar", "case_sensitive", 1) modparam("registrar", "append_branches", 0) modparam("registrar", "use_path", 1) modparam("registrar", "path_mode", 0) modparam("registrar", "path_use_received", 1) modparam("registrar", "path_check_local", 1) modparam("registrar", "max_contacts", 1) ``` usrloc ``` modparam("usrloc", "db_mode", 0) modparam("usrloc", "use_domain", 1) modparam("usrloc", "timer_interval", 60) modparam("usrloc", "timer_procs", 4) modparam("usrloc", "nat_bflag", 6) ```
nathelper ``` modparam("nathelper", "natping_interval", 20) modparam("nathelper", "natping_processes", 4) modparam("nathelper", "ping_nated_only", 0) modparam("nathelper", "sipping_from", "sip:keepalive@example.com") modparam("nathelper", "sipping_method", "OPTIONS") modparam("nathelper", "sipping_bflag", 6) modparam("nathelper", "force_socket", "10.7.0.190:5060") modparam("nathelper", "udpping_from_path", 1) ```
Nathelper force_socket is set to a socket that kamailio is listening on: ``` listen=udp:10.7.0.190:5060 ```
An example registered user we are trying to ping: kamctl ul show ``` { "jsonrpc": "2.0", "result": { "Domain": "location", "Size": 1024, "AoRs": [{ "Info": { "AoR": "example_user@example.com", "HashID": -1389656423, "Contacts": [{ "Contact": { "Address": "sip:example_user@212.2.172.228:39808;rinstance=ed8aa63e90f53e97;transport=UDP", "Expires": 66, "Q": -1, "Call-ID": "wE-GD4GzkrtuDAJUBJf1Jg..", "CSeq": 58, "User-Agent": "Z 3.15.40006 rv2.8.20", "Received": "sip:212.2.172.228:39808", "Path": "sip:10.7.0.186:5062;lr;received=sip:212.2.172.228:39808", "State": "CS_NEW", "Flags": 0, "CFlags": 64, "Socket": "udp:10.7.0.190:5060", "Methods": -1, "Ruid": "uloc-2-59fa1f9d-714-17", "Instance": "[not set]", "Reg-Id": 0, "Last-Keepalive": 1509615136, "Last-Modified": 1509615136 } }] } } ], "Stats": { "Records": 1, "Max-Slots": 1 } }, "id": 4836 } ``` #### Reproduction
With the above settings, when I do *not* have a default route set for the host then I encounter the following error in the logs when attempting to ping:
``` Nov 1 18:57:29 localhost kam_registrar_1[2793]: ERROR: nathelper [nathelper.c:2073]: nh_timer(): send_raw from path failed Nov 1 18:57:49 localhost kam_registrar_1[2793]: ERROR: nathelper [nathelper.c:2073]: nh_timer(): send_raw from path failed ```
If I comment out the "udpping_from_path" module parameter and restart kamailio, again, with *no* default route set, I get the following error:
``` Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: <core> [core/udp_server.c:591]: udp_send(): raw_iphdr_udp4_send(12,0x7f5be79587e6,4,...,212.2.172.228:39808,1500): Network is unreachable(101) Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: nathelper [nathelper.c:2077]: nh_timer(): udp_send failed ```
If I add a default route to the system, then the OPTIONS pings are able to flow. See example traffic in "SIP Traffic" section below.
#### Log Messages
Shown in description and troubleshooting sections
#### SIP Traffic
When a default route is set, successful options messages are sent as below.
``` U 2017/11/02 07:50:54.191717 10.7.0.190:5060 -> 10.7.0.186:5062
OPTIONS sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP SIP/2.0. Via: SIP/2.0/UDP 10.7.0.190:5060;branch=z9hG4bK5526436. Route: sip:10.7.0.186:5062;lr;received=sip:212.2.172.228:39808. From: sip:keepalive@example.com;tag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. To: sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP. Call-ID: d9993f61-a1fd4752-73cdc76@10.7.0.190. CSeq: 1 OPTIONS. Content-Length: 0. .
U 2017/11/02 07:50:54.192896 194.213.29.33:5062 -> 212.2.172.228:39808
OPTIONS sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP SIP/2.0. Max-Forwards: 10. Record-Route: sip:194.213.29.33:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Record-Route: sip:10.7.0.186:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Via: SIP/2.0/UDP 194.213.29.33:5062;branch=z9hG4bK13ab.9ccb0733fcecc331893d95f2e09485ee.0. Via: SIP/2.0/UDP 10.7.0.190:5060;rport=5060;branch=z9hG4bK5526436. From: sip:keepalive@example.com;tag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. To: sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP. Call-ID: d9993f61-a1fd4752-73cdc76@10.7.0.190. CSeq: 1 OPTIONS. Content-Length: 0. .
U 2017/11/02 07:50:54.248234 212.2.172.228:39808 -> 194.213.29.33:5062
SIP/2.0 200 OK. Via: SIP/2.0/UDP 194.213.29.33:5062;branch=z9hG4bK13ab.9ccb0733fcecc331893d95f2e09485ee.0. Via: SIP/2.0/UDP 10.7.0.190:5060;rport=5060;branch=z9hG4bK5526436. Record-Route: sip:194.213.29.33:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Record-Route: sip:10.7.0.186:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Contact: sip:192.168.1.64:39808. To: sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP;tag=726ffa30. From: sip:keepalive@example.com;tag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Call-ID: d9993f61-a1fd4752-73cdc76@10.7.0.190. CSeq: 1 OPTIONS. Accept: application/sdp, application/sdp. Accept-Language: en. Allow: INVITE, ACK, CANCEL, BYE, NOTIFY, REFER, MESSAGE, OPTIONS, INFO, SUBSCRIBE. Supported: replaces, norefersub, extended-refer, timer, outbound, path, X-cisco-serviceuri. User-Agent: Z 3.15.40006 rv2.8.20. Allow-Events: presence, kpml, talk. Content-Length: 0. .
U 2017/11/02 07:50:54.248922 10.7.0.186:5062 -> 10.7.0.190:5060
SIP/2.0 200 OK. Via: SIP/2.0/UDP 10.7.0.190:5060;rport=5060;branch=z9hG4bK5526436. Record-Route: sip:194.213.29.33:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Record-Route: sip:10.7.0.186:5062;r2=on;lr;ftag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Contact: sip:192.168.1.64:39808. To: sip:example_user@212.2.172.228:39808;rinstance=d74acdb581467154;transport=UDP;tag=726ffa30. From: sip:keepalive@example.com;tag=uloc-2-59fa1f9d-711-3-9968b2da-c2c36781. Call-ID: d9993f61-a1fd4752-73cdc76@10.7.0.190. CSeq: 1 OPTIONS. Accept: application/sdp, application/sdp. Accept-Language: en. Allow: INVITE, ACK, CANCEL, BYE, NOTIFY, REFER, MESSAGE, OPTIONS, INFO, SUBSCRIBE. Supported: replaces, norefersub, extended-refer, timer, outbound, path, X-cisco-serviceuri. User-Agent: Z 3.15.40006 rv2.8.20. Allow-Events: presence, kpml, talk. Content-Length: 0. . ```
### Possible Solutions
In this case I added a default route as a work-around, however, I don't think that this should be the "fix"
### Additional Information
* **Kamailio Version**: ``` version: kamailio 5.0.4 (x86_64/linux) flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled on 10:57:22 Oct 26 2017 with gcc 4.8.5 ```
* **Operating System**: ``` CentOS Linux release 7.4.1708 (Core) Linux localhost 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux ```
Socket can be forced when sip ping is used. You have to set appropriate branch flag per register to match the modparam value.
You have other settings here, so when you set to send ping from path, the 4-zero-bytes ping is done and it creates UDP raw sockets to fake the source IP. In this case the local socket is irrelevant, because the source ip is set to something else. It is the kernel creating a raw socket for sending out.
I am not sure what can be done here, it seems to be just like it is supposed to happen. Do you want an option to not print the error messages?
Thanks for the opinion. I originally was not running with "udpping_from_path" (it was disabled). Even with it disabled and only using SIP OPTIONS ping method the following log was generated: ``` Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: <core> [core/udp_server.c:591]: udp_send(): raw_iphdr_udp4_send(12,0x7f5be79587e6,4,...,212.2.172.228:39808,1500): Network is unreachable(101) Nov 1 18:58:01 localhost kam_registrar_1[2896]: ERROR: nathelper [nathelper.c:2077]: nh_timer(): udp_send failed ``` Lets progress this issue assuming I am not using "udpping_from_path" and only using SIP OPTIONS ping method.
I guess what I am asking here is that, if kamailio has a socket defined on the local network, and its $du is on the same network, then, nathelper should be able to send the message. However, in this case, it cannot. It looks like it is trying to work out which interface to send the destination contact (external address: 212.2.172.228) to, but, because we are behind a proxy on a private lan, then kamailio needs to send via the PATH definition, in this case all OPTIONS messages will go out via an address on the local lan.
In my humble opinion, we should not need to define a default gateway so that kamailio can resolve which interface to use (in this case the wrong one (10.6.0.x because the default gateway is a 10.6.0.x address), only for nathelper/registrar to realize that it needs to use the PATH parameter for sending the messages, and only then relay the message.
I'm not sure if I am explaining myself correctly, please let me know if I am not being clear in my explanation and I'll try to explain better.
I was looking at the error messages related to send_raw.
So, if the raw socket case is dismissed, now this seems related to the force socket not be used? If yes, then this issue should be closed, because you opened another one for that specific problem, and continue there.
ahh okay, if you think we can close this one then, sure, we can close it. Originally I was not using udp ping's I just tried it out to see if it would work in this case.
I am interested in getting the SIP method pings working (OPTIONS/INFO etc), if the force_socket ticket is the right place to progress this, then lets close this one.
If we can get force_socket to work, then I can test it again by removing the default gateway and seeing if kamailio is able to send a keep-alive out.
Closed #1297.
Kamailio is not managing at all routing rules at IP layer. If the right socket is used, but routing fails, nothing can be fixed in kamailio, it just does libc udp send.
Let's close this one and continue on #1298 for sorting out the use of a specific socket.
-
00Asgaroth00 -
Daniel-Constantin Mierla