On 9/16/15 4:00 AM, Sebastian Damm wrote:
Hi,
we have a load balancer which is handling a lot of SIP traffic all day. There's always 20-40 Mbit SIP traffic going through. From time to time we see in our logs messages like these:
Sep 16 09:46:28 ecker /usr/sbin/kamailio[25505]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7f2d9d6b3ce0,1321,0,46.237.225.126:5060 http://46.237.225.126:5060,16): Operation not permitted(1) Sep 16 09:46:38 ecker /usr/sbin/kamailio[25194]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7efc982b8fc8,420,0,82.113.121.183:35794 http://82.113.121.183:35794,16): Operation not permitted(1) Sep 16 09:46:40 ecker /usr/sbin/kamailio[25505]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7f2d9d6b3ce0,1338,0,5.158.137.9:55067 http://5.158.137.9:55067,16): Operation not permitted(1) Sep 16 09:46:44 ecker /usr/sbin/kamailio[25183]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7efc982d9f48,450,0,178.165.131.197:37515 http://178.165.131.197:37515,16): Operation not permitted(1) Sep 16 09:46:49 ecker /usr/sbin/kamailio[25643]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7f93fb624530,496,0,172.56.7.69:25643 http://172.56.7.69:25643,16): Operation not permitted(1) Sep 16 09:46:55 ecker /usr/sbin/kamailio[25335]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7f41632cda98,598,0,80.215.234.139:3396 http://80.215.234.139:3396,16): Operation not permitted(1) Sep 16 09:46:56 ecker /usr/sbin/kamailio[25345]: ERROR: <core> [udp_server.c:591]: udp_send(): ERROR: udp_send: sendto(sock,0x7f41632f4840,459,0,94.197.120.191:8225 http://94.197.120.191:8225,16): Operation not permitted(1)
I know that these messages can be produced by iptables blocking the outbound traffic. But our outbound chain looks basically like this:
root@ecker:~# iptables-save | grep OUTPUT :OUTPUT DROP [0:0] -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -j ACCEPT -A OUTPUT -m state --state INVALID -j DROP -A OUTPUT -o lo -m state --state NEW -j ACCEPT -A OUTPUT -o lo -j ACCEPT -A OUTPUT -m state --state NEW -j ACCEPT
We don't have the nf_ct_sip module loaded, syslog doesn't say anything, and even clearing all iptables rules doesn't eliminate those errors.
Has anyone ever seen this? It looks like a load thing, because at weekends there are significantly less errors.
Try to see if the transmit queue is high (third column) using netstat. You always want that to drop down to zero fast when it goes up. If it starts to grow, you definitely have a problem between the application and operating system.
# netstat -aupn | grep kama udp 0 0 23.239.16.141:50963 23.239.16.141:22555 ESTABLISHED 26101/kamailio udp 0 0 23.239.16.141:50195 198.58.110.117:22555 ESTABLISHED 26081/kamailio udp 0 0 23.239.16.141:42772 198.58.110.117:22555 ESTABLISHED 26122/kamailio udp 0 0 23.239.16.141:49046 23.239.16.141:22555 ESTABLISHED 26112/kamailio udp 0 0 23.239.16.141:49181 198.58.110.117:22555 ESTABLISHED 26118/kamailio udp 0 0 23.239.16.141:36898 23.239.16.141:22555 ESTABLISHED 26122/kamailio udp 0 0 23.239.16.141:59813 23.239.16.141:22555 ESTABLISHED 26120/kamailio udp 0 0 23.239.16.141:36016 23.239.16.141:22555 ESTABLISHED 26110/kamailio udp 0 0 23.239.16.141:47541 198.58.110.117:22555 ESTABLISHED 26103/kamailio udp 0 0 23.239.16.141:60727 23.239.16.141:22555 ESTABLISHED 26096/kamailio udp 0 0 23.239.16.141:52282 198.58.110.117:22555 ESTABLISHED 26120/kamailio udp 0 0 23.239.16.141:57275 198.58.110.117:22555 ESTABLISHED 26114/kamailio udp 0 0 23.239.16.141:51137 23.239.16.141:22555 ESTABLISHED 26114/kamailio udp 0 0 23.239.16.141:53574 23.239.16.141:22555 ESTABLISHED 26095/kamailio udp 0 0 23.239.16.141:5065 0.0.0.0:* 26081/kamailio
Thanks, Sebastian
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users