Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
10(36629) DEBUG: tm [t_reply.c:659]: DEBUG: reply sent out. buf=0x832c7ac: SIP/2.0 100 trying -..., shmem=0x28d97c7c: SIP/2.0 100 trying - 10(36629) DEBUG: tm [t_reply.c:669]: DEBUG: _reply_light: finished 10(36629) DEBUG: <core> [forward.c:213]: DEBUG: get_out_socket: socket determined: 0x833ddc0 10(36629) DEBUG: <core> [msg_translator.c:457]: clen_builder: content-length: 553 (553) 10(36629) DEBUG: <core> [msg_translator.c:204]: check_via_address(10.200.245.200, 10.200.245.200, 0) 10(36629) DEBUG: <core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 10(36629) DEBUG: <core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.198.3.100 10(36629) DEBUG: <core> [tcp_main.c:1081]: tcpconn_new: on port 5060, type 2 10(36629) DEBUG: <core> [tcp_main.c:1382]: tcpconn_add: hashes: 0:0:0, 1 10(36629) ERROR: <core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected 10(36629) ERROR: <core> [tcp_main.c:2754]: 10.198.3.100:5060: connect & send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG: <core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
We tested two different variants: one when defined Lync in dr_gateways table as 10.198.3.100:5068;transport=tcp
And then, tried global options udp_mtu=1300 udp_mtu_try_proto=TCP
As soon as these options were enabled, all other gateways became unreachable with the same error as above.
Could you please to point me out, where I'm wrong?
fbsd-kam# /usr/local/sbin/kamailio -V version: kamailio 3.1.5 (i386/freebsd) ec672a flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, PKG_SIZE 4MB poll method support: poll, select, kqueue. id: ec672a compiled on 11:02:58 Nov 15 2011 with gcc 4.2.1
FreeBSD fbsd-kam.dprs-consulting.com 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Jul 3 17:49:13 GMT 2011 root@fbsd-kam.dprs-consulting.com:/usr/obj/usr/src/sys/GENERIC i386
Routing script could be sent on request.
Thank you very much!
WBR Dimon
Hello,
you haven't disabled tcp support, right? Can you do 'kamctl ps' and see if you have tcp workers?
I see the error is related to 10.198.3.100:5060 :
10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected
What is listening on this ip and port? In drouting table you have 10.198.3.100:5068;transport=tcp -- a different port.
Then, do you need to enable and play with mtu params? This is for auto-switching from UDP to TCP when size of SIP message exceeds udp_mtu value in bytes, done by SIP server, but if the other party is not listening on tcp, then it won't work (reason is disabled by default) -- note that this switching will be done for all sip messages.
Cheers, Daniel
On 11/15/11 11:43 AM, Dmitry Petrakoff wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
10(36629) DEBUG: tm [t_reply.c:659]: DEBUG: reply sent out. buf=0x832c7ac: SIP/2.0 100 trying -..., shmem=0x28d97c7c: SIP/2.0 100 trying - 10(36629) DEBUG: tm [t_reply.c:669]: DEBUG: _reply_light: finished 10(36629) DEBUG:<core> [forward.c:213]: DEBUG: get_out_socket: socket determined: 0x833ddc0 10(36629) DEBUG:<core> [msg_translator.c:457]: clen_builder: content-length: 553 (553) 10(36629) DEBUG:<core> [msg_translator.c:204]: check_via_address(10.200.245.200, 10.200.245.200, 0) 10(36629) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 10(36629) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.198.3.100 10(36629) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5060, type 2 10(36629) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 0:0:0, 1 10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected 10(36629) ERROR:<core> [tcp_main.c:2754]: 10.198.3.100:5060: connect& send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
We tested two different variants: one when defined Lync in dr_gateways table as 10.198.3.100:5068;transport=tcp
And then, tried global options udp_mtu=1300 udp_mtu_try_proto=TCP
As soon as these options were enabled, all other gateways became unreachable with the same error as above.
Could you please to point me out, where I'm wrong?
fbsd-kam# /usr/local/sbin/kamailio -V version: kamailio 3.1.5 (i386/freebsd) ec672a flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, PKG_SIZE 4MB poll method support: poll, select, kqueue. id: ec672a compiled on 11:02:58 Nov 15 2011 with gcc 4.2.1
FreeBSD fbsd-kam.dprs-consulting.com 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Jul 3 17:49:13 GMT 2011 root@fbsd-kam.dprs-consulting.com:/usr/obj/usr/src/sys/GENERIC i386
Routing script could be sent on request.
Thank you very much!
WBR Dimon
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Daniel-Constantin!
Many thanks for your reply Sorry for the wrong output from DEBUG. I took it when such directives were set in config:
#disable_tcp=yes udp_mtu=1300 udp_mtu_try_proto=TCP
Actually, that host 10.198.3.100 works just fine via UDP.
But when those directives were enabled, next INVITE has gone there via TCP and the error was the same as with Lync.
More over. Today I run dispatcher module as "stupid" pinger. And again, faced with the same error.
Now it is output from kamctl ps:
Process:: ID=0 PID=40677 Type=attendant Process:: ID=1 PID=40678 Type=udp receiver child=0 sock=1.2.3.4:5060 Process:: ID=2 PID=40679 Type=udp receiver child=1 sock=1.2.3.4:5060 Process:: ID=3 PID=40680 Type=udp receiver child=2 sock=1.2.3.4:5060 Process:: ID=4 PID=40681 Type=udp receiver child=3 sock=1.2.3.4:5060 Process:: ID=5 PID=40682 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=6 PID=40683 Type=udp receiver child=1 sock=127.0.0.1:5060 Process:: ID=7 PID=40684 Type=udp receiver child=2 sock=127.0.0.1:5060 Process:: ID=8 PID=40685 Type=udp receiver child=3 sock=127.0.0.1:5060 Process:: ID=9 PID=40686 Type=udp receiver child=0 sock=10.200.245.250:5060 Process:: ID=10 PID=40687 Type=udp receiver child=1 sock=10.200.245.250:5060 Process:: ID=11 PID=40688 Type=udp receiver child=2 sock=10.200.245.250:5060 Process:: ID=12 PID=40689 Type=udp receiver child=3 sock=10.200.245.250:5060 Process:: ID=13 PID=40690 Type=slow timer Process:: ID=14 PID=40691 Type=timer Process:: ID=15 PID=40692 Type=MI FIFO Process:: ID=16 PID=40693 Type=ctl handler Process:: ID=17 PID=40694 Type=TIMER NH Process:: ID=18 PID=40695 Type=tcp receiver child=0 Process:: ID=19 PID=40696 Type=tcp receiver child=1 Process:: ID=20 PID=40697 Type=tcp receiver child=2 Process:: ID=21 PID=40698 Type=tcp receiver child=3 Process:: ID=22 PID=40699 Type=tcp main process
As you can see, TCP is present and working for receiving. And we were able to receive INVITE from Lync via TCP.
Now, part of newly added dispatcher functionality:
loadmodule "dispatcher.so" modparam("dispatcher", "db_url", DBURL); modparam("dispatcher", "ds_ping_from", "sip:openser@10.200.245.250"); modparam("dispatcher", "ds_ping_interval", 5); modparam("dispatcher", "ds_probing_mode", 1);
and table:
mysql> select * from dispatcher; +----+-------+---------------------------------------+-------+----------+-------+-------------+ | id | setid | destination | flags | priority | attrs | description | +----+-------+---------------------------------------+-------+----------+-------+-------------+ | 1 | 1 | sip:10.200.253.186:5068;transport=tcp | 0 | 0 | | Lync-Test | +----+-------+---------------------------------------+-------+----------+-------+-------------+
corresponding tcp options are:
#disable_tcp=yes tcp_children=4
and another, probably important, option which I forgot to inform about:
mhomed=1
And what I'm getting.
From syslog
Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: <core> [tcp_mai n.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: <core> [tcp_mai n.c:2754]: 10.200.253.186:5068: connect & send for 0x28db87f8 failed: Socket is not connected (57) Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [../../forwa rd.h:170]: msg_send: ERROR: tcp_send failed Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [uac.c:496]: t_uac: Attempt to send to precreated request failed
When run with DEBUG
13(67161) DEBUG: dispatcher [dispatch.c:2305]: probing set #1, URI sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:242]: DEBUG:tm:t_uac: next_hop=sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:181]: DEBUG: dlg2hash: 62915 13(67161) DEBUG: <core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 13(67161) DEBUG: <core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.200.253.186 13(67161) DEBUG: <core> [tcp_main.c:1081]: tcpconn_new: on port 5068, type 2 13(67161) DEBUG: <core> [tcp_main.c:1382]: tcpconn_add: hashes: 3509:0:0, 4 13(67161) ERROR: <core> [tcp_main.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected 13(67161) ERROR: <core> [tcp_main.c:2754]: 10.200.253.186:5068: connect & send for 0x28dbc308 failed: Socket is not connected (57) 13(67161) DEBUG: <core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28dbc308 (4, -1) flags 0060 13(67161) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Other options from config:
fork=yes children=4 user="kamailio" group="kamailio" server_header="DPRS's SIP proxy" server_signature=no
So as you can see, here is the same error.
Many thanks in advance!
WBR Dimon
On 16.11.11 11:46, Daniel-Constantin Mierla wrote:
Hello,
you haven't disabled tcp support, right? Can you do 'kamctl ps' and see if you have tcp workers?
I see the error is related to 10.198.3.100:5060 :
10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected
What is listening on this ip and port? In drouting table you have 10.198.3.100:5068;transport=tcp -- a different port.
Then, do you need to enable and play with mtu params? This is for auto-switching from UDP to TCP when size of SIP message exceeds udp_mtu value in bytes, done by SIP server, but if the other party is not listening on tcp, then it won't work (reason is disabled by default) -- note that this switching will be done for all sip messages.
Cheers, Daniel
On 11/15/11 11:43 AM, Dmitry Petrakoff wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
10(36629) DEBUG: tm [t_reply.c:659]: DEBUG: reply sent out. buf=0x832c7ac: SIP/2.0 100 trying -..., shmem=0x28d97c7c: SIP/2.0 100 trying - 10(36629) DEBUG: tm [t_reply.c:669]: DEBUG: _reply_light: finished 10(36629) DEBUG:<core> [forward.c:213]: DEBUG: get_out_socket: socket determined: 0x833ddc0 10(36629) DEBUG:<core> [msg_translator.c:457]: clen_builder: content-length: 553 (553) 10(36629) DEBUG:<core> [msg_translator.c:204]: check_via_address(10.200.245.200, 10.200.245.200, 0) 10(36629) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 10(36629) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.198.3.100 10(36629) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5060, type 2 10(36629) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 0:0:0, 1 10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected 10(36629) ERROR:<core> [tcp_main.c:2754]: 10.198.3.100:5060: connect& send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
We tested two different variants: one when defined Lync in dr_gateways table as 10.198.3.100:5068;transport=tcp
And then, tried global options udp_mtu=1300 udp_mtu_try_proto=TCP
As soon as these options were enabled, all other gateways became unreachable with the same error as above.
Could you please to point me out, where I'm wrong?
fbsd-kam# /usr/local/sbin/kamailio -V version: kamailio 3.1.5 (i386/freebsd) ec672a flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, PKG_SIZE 4MB poll method support: poll, select, kqueue. id: ec672a compiled on 11:02:58 Nov 15 2011 with gcc 4.2.1
FreeBSD fbsd-kam.dprs-consulting.com 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Jul 3 17:49:13 GMT 2011 root@fbsd-kam.dprs-consulting.com:/usr/obj/usr/src/sys/GENERIC i386
Routing script could be sent on request.
Thank you very much!
WBR Dimon
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hello,
On 11/16/11 9:15 AM, Dmitry Petrakoff wrote:
Hi Daniel-Constantin!
Many thanks for your reply Sorry for the wrong output from DEBUG. I took it when such directives were set in config:
#disable_tcp=yes udp_mtu=1300 udp_mtu_try_proto=TCP
Actually, that host 10.198.3.100 works just fine via UDP.
But when those directives were enabled, next INVITE has gone there via TCP and the error was the same as with Lync.
yes, the question was if the host is able to take tcp since the conversion from tcp to udp is done by the sip server automatically when the size of the message exceeds udp mtu.
If you have another kamailio listening on tcp, are you able to send traffic to it over tcp?
Cheers, Daniel
More over. Today I run dispatcher module as "stupid" pinger. And again, faced with the same error.
Now it is output from kamctl ps:
Process:: ID=0 PID=40677 Type=attendant Process:: ID=1 PID=40678 Type=udp receiver child=0 sock=1.2.3.4:5060 Process:: ID=2 PID=40679 Type=udp receiver child=1 sock=1.2.3.4:5060 Process:: ID=3 PID=40680 Type=udp receiver child=2 sock=1.2.3.4:5060 Process:: ID=4 PID=40681 Type=udp receiver child=3 sock=1.2.3.4:5060 Process:: ID=5 PID=40682 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=6 PID=40683 Type=udp receiver child=1 sock=127.0.0.1:5060 Process:: ID=7 PID=40684 Type=udp receiver child=2 sock=127.0.0.1:5060 Process:: ID=8 PID=40685 Type=udp receiver child=3 sock=127.0.0.1:5060 Process:: ID=9 PID=40686 Type=udp receiver child=0 sock=10.200.245.250:5060 Process:: ID=10 PID=40687 Type=udp receiver child=1 sock=10.200.245.250:5060 Process:: ID=11 PID=40688 Type=udp receiver child=2 sock=10.200.245.250:5060 Process:: ID=12 PID=40689 Type=udp receiver child=3 sock=10.200.245.250:5060 Process:: ID=13 PID=40690 Type=slow timer Process:: ID=14 PID=40691 Type=timer Process:: ID=15 PID=40692 Type=MI FIFO Process:: ID=16 PID=40693 Type=ctl handler Process:: ID=17 PID=40694 Type=TIMER NH Process:: ID=18 PID=40695 Type=tcp receiver child=0 Process:: ID=19 PID=40696 Type=tcp receiver child=1 Process:: ID=20 PID=40697 Type=tcp receiver child=2 Process:: ID=21 PID=40698 Type=tcp receiver child=3 Process:: ID=22 PID=40699 Type=tcp main process
As you can see, TCP is present and working for receiving. And we were able to receive INVITE from Lync via TCP.
Now, part of newly added dispatcher functionality:
loadmodule "dispatcher.so" modparam("dispatcher", "db_url", DBURL); modparam("dispatcher", "ds_ping_from", "sip:openser@10.200.245.250"); modparam("dispatcher", "ds_ping_interval", 5); modparam("dispatcher", "ds_probing_mode", 1);
and table:
mysql> select * from dispatcher; +----+-------+---------------------------------------+-------+----------+-------+-------------+ | id | setid | destination | flags | priority | attrs | description | +----+-------+---------------------------------------+-------+----------+-------+-------------+ | 1 | 1 | sip:10.200.253.186:5068;transport=tcp | 0 | 0 | | Lync-Test | +----+-------+---------------------------------------+-------+----------+-------+-------------+
corresponding tcp options are:
#disable_tcp=yes tcp_children=4
and another, probably important, option which I forgot to inform about:
mhomed=1
And what I'm getting.
From syslog
Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR:<core> [tcp_mai n.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR:<core> [tcp_mai n.c:2754]: 10.200.253.186:5068: connect& send for 0x28db87f8 failed: Socket is not connected (57) Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [../../forwa rd.h:170]: msg_send: ERROR: tcp_send failed Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [uac.c:496]: t_uac: Attempt to send to precreated request failed
When run with DEBUG
13(67161) DEBUG: dispatcher [dispatch.c:2305]: probing set #1, URI sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:242]: DEBUG:tm:t_uac: next_hop=sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:181]: DEBUG: dlg2hash: 62915 13(67161) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 13(67161) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.200.253.186 13(67161) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5068, type 2 13(67161) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 3509:0:0, 4 13(67161) ERROR:<core> [tcp_main.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected 13(67161) ERROR:<core> [tcp_main.c:2754]: 10.200.253.186:5068: connect & send for 0x28dbc308 failed: Socket is not connected (57) 13(67161) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28dbc308 (4, -1) flags 0060 13(67161) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Other options from config:
fork=yes children=4 user="kamailio" group="kamailio" server_header="DPRS's SIP proxy" server_signature=no
So as you can see, here is the same error.
Many thanks in advance!
WBR Dimon
On 16.11.11 11:46, Daniel-Constantin Mierla wrote:
Hello,
you haven't disabled tcp support, right? Can you do 'kamctl ps' and see if you have tcp workers?
I see the error is related to 10.198.3.100:5060 :
10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected
What is listening on this ip and port? In drouting table you have 10.198.3.100:5068;transport=tcp -- a different port.
Then, do you need to enable and play with mtu params? This is for auto-switching from UDP to TCP when size of SIP message exceeds udp_mtu value in bytes, done by SIP server, but if the other party is not listening on tcp, then it won't work (reason is disabled by default) -- note that this switching will be done for all sip messages.
Cheers, Daniel
On 11/15/11 11:43 AM, Dmitry Petrakoff wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
10(36629) DEBUG: tm [t_reply.c:659]: DEBUG: reply sent out. buf=0x832c7ac: SIP/2.0 100 trying -..., shmem=0x28d97c7c: SIP/2.0 100 trying - 10(36629) DEBUG: tm [t_reply.c:669]: DEBUG: _reply_light: finished 10(36629) DEBUG:<core> [forward.c:213]: DEBUG: get_out_socket: socket determined: 0x833ddc0 10(36629) DEBUG:<core> [msg_translator.c:457]: clen_builder: content-length: 553 (553) 10(36629) DEBUG:<core> [msg_translator.c:204]: check_via_address(10.200.245.200, 10.200.245.200, 0) 10(36629) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 10(36629) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.198.3.100 10(36629) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5060, type 2 10(36629) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 0:0:0, 1 10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected 10(36629) ERROR:<core> [tcp_main.c:2754]: 10.198.3.100:5060: connect& send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
We tested two different variants: one when defined Lync in dr_gateways table as 10.198.3.100:5068;transport=tcp
And then, tried global options udp_mtu=1300 udp_mtu_try_proto=TCP
As soon as these options were enabled, all other gateways became unreachable with the same error as above.
Could you please to point me out, where I'm wrong?
fbsd-kam# /usr/local/sbin/kamailio -V version: kamailio 3.1.5 (i386/freebsd) ec672a flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, PKG_SIZE 4MB poll method support: poll, select, kqueue. id: ec672a compiled on 11:02:58 Nov 15 2011 with gcc 4.2.1
FreeBSD fbsd-kam.dprs-consulting.com 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Jul 3 17:49:13 GMT 2011 root@fbsd-kam.dprs-consulting.com:/usr/obj/usr/src/sys/GENERIC i386
Routing script could be sent on request.
Thank you very much!
WBR Dimon
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hi Daniel-Constantin!
On 17.11.11 10:56, Daniel-Constantin Mierla wrote:
Hello,
On 11/16/11 9:15 AM, Dmitry Petrakoff wrote:
Hi Daniel-Constantin!
Many thanks for your reply Sorry for the wrong output from DEBUG. I took it when such directives were set in config:
#disable_tcp=yes udp_mtu=1300 udp_mtu_try_proto=TCP
Actually, that host 10.198.3.100 works just fine via UDP.
But when those directives were enabled, next INVITE has gone there via TCP and the error was the same as with Lync.
yes, the question was if the host is able to take tcp since the conversion from tcp to udp is done by the sip server automatically when the size of the message exceeds udp mtu.
When both these udp_mtu and udp_mtu_try_proto are set, when server receives INVITE from UAC via UDP with size more then set in udp_mtu, then he tries to forward this INVITE to other party using TCP. But this attempt generates the same error when sending as you may see in my previous posts.
From the other side, if server getting INVITE via TCP and forwarding it
via UDP, everything works good. I.e. server listens on TCP and can to respond for incoming TCP requests. But can't create a socket ( under FreeBSD 8 at least ) to initiate TCP session.
Under tcpdump I see fisrt SYN from kamailio to destination; destination reply with SYNACK but kamailio sends RESET instead of ACK:
09:44:41.869687 IP 10.200.245.250.10274 > 10.200.253.186.5068: Flags [S], seq 3232460831, win 65535, options [mss 1460,nop,wscale 3,sackOK,TS val 519365170 ecr 0], length 0 09:44:41.870304 IP 10.200.253.186.5068 > 10.200.245.250.10274: Flags [S.], seq 1977596128, ack 3232460832, win 8192, options [mss 1460,nop,wscale 8,sackOK,TS val 7346469 ecr 519365170], length 0 09:44:41.870364 IP 10.200.245.250.10274 > 10.200.253.186.5068: Flags [R], seq 3232460832, win 0, length 0
If you have another kamailio listening on tcp, are you able to send traffic to it over tcp?
Yes we've set up quickly Kamalio-3.1.5 under Ubuntu ( Linux ubu-kam 2.6.32-33-generic-pae #72-Ubuntu SMP Fri Jul 29 22:06:29 UTC 2011 i686 GNU/Linux Ubuntu 10.04.3 LTS ) with just dispatcher's pinging functionality for testing and it works. Here is DEBUG output: 9(1415) DEBUG: <core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 9(1415) DEBUG: <core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.200.253.186 9(1415) DEBUG: <core> [tcp_main.c:1081]: tcpconn_new: on port 5068, type 2 9(1415) DEBUG: <core> [tcp_main.c:1382]: tcpconn_add: hashes: 3509:0:0, 2 9(1415) DEBUG: <core> [tcp_main.c:2700]: pending write on new connection 0xb5ac3f5c (-1/305 bytes written) 17(1423) DEBUG: <core> [tcp_main.c:3583]: handle_ser_child: read response= b5ac3f5c, 5, fd 29 from 9 (1415) 17(1423) DEBUG: <core> [io_wait.h:390]: DBG: io_watch_add(0x82430c0, 29, 2, 0xb5ac3f5c), fd_no=22 17(1423) DEBUG: <core> [io_wait.h:796]: DBG: io_watch_chg (0x82430c0, 29, 0x1, 0xffffffff) fd_no=23 called 17(1423) DEBUG: <core> [io_wait.h:617]: DBG: io_watch_del (0x82430c0, 29, -1, 0x0) fd_no=23 called 17(1423) DEBUG: <core> [tcp_main.c:4204]: tcp: DBG: sending to child, events 2001 17(1423) DEBUG: <core> [tcp_main.c:3889]: send2child: to tcp child 1 14(1420), 0xb5ac3f5c
And here is tcpdump:
14:44:10.534273 IP 10.200.251.6.53280 > 10.200.253.186.5068: Flags [S], seq 2844516733, win 5840, options [mss 1460,sackOK,TS val 600506 ecr 0,nop,wscale 6], length 0 14:44:10.534736 IP 10.200.253.186.5068 > 10.200.251.6.53280: Flags [S.], seq 100170775, ack 2844516734, win 8192, options [mss 1460,nop,wscale 8,sackOK,TS val 356125 ecr 600506], length 0 14:44:10.534793 IP 10.200.251.6.53280 > 10.200.253.186.5068: Flags [P.], seq 1:308, ack 1, win 92, options [nop,nop,TS val 600506 ecr 356125], length 307 14:44:10.536831 IP 10.200.253.186.5068 > 10.200.251.6.53280: Flags [F.], seq 1, ack 308, win 260, options [nop,nop,TS val 356125 ecr 600506], length 0 14:44:10.537004 IP 10.200.251.6.53280 > 10.200.253.186.5068: Flags [F.], seq 308, ack 2, win 92, options [nop,nop,TS val 600507 ecr 356125], length 0 14:44:10.537427 IP 10.200.253.186.5068 > 10.200.251.6.53280: Flags [.], ack 309, win 260, options [nop,nop,TS val 356126 ecr 600507], length 0
I.e. TCP sending under Linux works just fine.
As you can see, the error happens right after string where [tcp_main.c:1382]. Under FreeBSD we've got an error, but under Linux - not.
Unfortunately, I was not able to check this under OpenBSD-5.0 because can't compile kamailio at all. But that was a different story.
Thank you very much! I'm ready to provide any additional info you will need.
WBR Dimon
Cheers, Daniel
More over. Today I run dispatcher module as "stupid" pinger. And again, faced with the same error.
Now it is output from kamctl ps:
Process:: ID=0 PID=40677 Type=attendant Process:: ID=1 PID=40678 Type=udp receiver child=0 sock=1.2.3.4:5060 Process:: ID=2 PID=40679 Type=udp receiver child=1 sock=1.2.3.4:5060 Process:: ID=3 PID=40680 Type=udp receiver child=2 sock=1.2.3.4:5060 Process:: ID=4 PID=40681 Type=udp receiver child=3 sock=1.2.3.4:5060 Process:: ID=5 PID=40682 Type=udp receiver child=0 sock=127.0.0.1:5060 Process:: ID=6 PID=40683 Type=udp receiver child=1 sock=127.0.0.1:5060 Process:: ID=7 PID=40684 Type=udp receiver child=2 sock=127.0.0.1:5060 Process:: ID=8 PID=40685 Type=udp receiver child=3 sock=127.0.0.1:5060 Process:: ID=9 PID=40686 Type=udp receiver child=0 sock=10.200.245.250:5060 Process:: ID=10 PID=40687 Type=udp receiver child=1 sock=10.200.245.250:5060 Process:: ID=11 PID=40688 Type=udp receiver child=2 sock=10.200.245.250:5060 Process:: ID=12 PID=40689 Type=udp receiver child=3 sock=10.200.245.250:5060 Process:: ID=13 PID=40690 Type=slow timer Process:: ID=14 PID=40691 Type=timer Process:: ID=15 PID=40692 Type=MI FIFO Process:: ID=16 PID=40693 Type=ctl handler Process:: ID=17 PID=40694 Type=TIMER NH Process:: ID=18 PID=40695 Type=tcp receiver child=0 Process:: ID=19 PID=40696 Type=tcp receiver child=1 Process:: ID=20 PID=40697 Type=tcp receiver child=2 Process:: ID=21 PID=40698 Type=tcp receiver child=3 Process:: ID=22 PID=40699 Type=tcp main process
As you can see, TCP is present and working for receiving. And we were able to receive INVITE from Lync via TCP.
Now, part of newly added dispatcher functionality:
loadmodule "dispatcher.so" modparam("dispatcher", "db_url", DBURL); modparam("dispatcher", "ds_ping_from", "sip:openser@10.200.245.250"); modparam("dispatcher", "ds_ping_interval", 5); modparam("dispatcher", "ds_probing_mode", 1);
and table:
mysql> select * from dispatcher; +----+-------+---------------------------------------+-------+----------+-------+-------------+
| id | setid | destination | flags | priority | attrs | description | +----+-------+---------------------------------------+-------+----------+-------+-------------+
| 1 | 1 | sip:10.200.253.186:5068;transport=tcp | 0 | 0 | | Lync-Test | +----+-------+---------------------------------------+-------+----------+-------+-------------+
corresponding tcp options are:
#disable_tcp=yes tcp_children=4
and another, probably important, option which I forgot to inform about:
mhomed=1
And what I'm getting.
From syslog
Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR:<core> [tcp_mai n.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR:<core> [tcp_mai n.c:2754]: 10.200.253.186:5068: connect& send for 0x28db87f8 failed: Socket is not connected (57) Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [../../forwa rd.h:170]: msg_send: ERROR: tcp_send failed Nov 16 11:07:04 fbsd-kam /usr/local/sbin/kamailio[60981]: ERROR: tm [uac.c:496]: t_uac: Attempt to send to precreated request failed
When run with DEBUG
13(67161) DEBUG: dispatcher [dispatch.c:2305]: probing set #1, URI sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:242]: DEBUG:tm:t_uac: next_hop=sip:10.200.253.186:5068;transport=tcp 13(67161) DEBUG: tm [uac.c:181]: DEBUG: dlg2hash: 62915 13(67161) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 13(67161) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.200.253.186 13(67161) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5068, type 2 13(67161) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 3509:0:0, 4 13(67161) ERROR:<core> [tcp_main.c:2748]: connect 10.200.253.186:5068 failed Socket is not connected 13(67161) ERROR:<core> [tcp_main.c:2754]: 10.200.253.186:5068: connect & send for 0x28dbc308 failed: Socket is not connected (57) 13(67161) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28dbc308 (4, -1) flags 0060 13(67161) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Other options from config:
fork=yes children=4 user="kamailio" group="kamailio" server_header="DPRS's SIP proxy" server_signature=no
So as you can see, here is the same error.
Many thanks in advance!
WBR Dimon
On 16.11.11 11:46, Daniel-Constantin Mierla wrote:
Hello,
you haven't disabled tcp support, right? Can you do 'kamctl ps' and see if you have tcp workers?
I see the error is related to 10.198.3.100:5060 :
10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected
What is listening on this ip and port? In drouting table you have 10.198.3.100:5068;transport=tcp -- a different port.
Then, do you need to enable and play with mtu params? This is for auto-switching from UDP to TCP when size of SIP message exceeds udp_mtu value in bytes, done by SIP server, but if the other party is not listening on tcp, then it won't work (reason is disabled by default) -- note that this switching will be done for all sip messages.
Cheers, Daniel
On 11/15/11 11:43 AM, Dmitry Petrakoff wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
10(36629) DEBUG: tm [t_reply.c:659]: DEBUG: reply sent out. buf=0x832c7ac: SIP/2.0 100 trying -..., shmem=0x28d97c7c: SIP/2.0 100 trying - 10(36629) DEBUG: tm [t_reply.c:669]: DEBUG: _reply_light: finished 10(36629) DEBUG:<core> [forward.c:213]: DEBUG: get_out_socket: socket determined: 0x833ddc0 10(36629) DEBUG:<core> [msg_translator.c:457]: clen_builder: content-length: 553 (553) 10(36629) DEBUG:<core> [msg_translator.c:204]: check_via_address(10.200.245.200, 10.200.245.200, 0) 10(36629) DEBUG:<core> [tcp_main.c:1824]: tcp_send: no open tcp connection found, opening new one 10(36629) DEBUG:<core> [ip_addr.c:247]: tcpconn_new: new tcp connection: 10.198.3.100 10(36629) DEBUG:<core> [tcp_main.c:1081]: tcpconn_new: on port 5060, type 2 10(36629) DEBUG:<core> [tcp_main.c:1382]: tcpconn_add: hashes: 0:0:0, 1 10(36629) ERROR:<core> [tcp_main.c:2748]: connect 10.198.3.100:5060 failed Socket is not connected 10(36629) ERROR:<core> [tcp_main.c:2754]: 10.198.3.100:5060: connect& send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG:<core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
We tested two different variants: one when defined Lync in dr_gateways table as 10.198.3.100:5068;transport=tcp
And then, tried global options udp_mtu=1300 udp_mtu_try_proto=TCP
As soon as these options were enabled, all other gateways became unreachable with the same error as above.
Could you please to point me out, where I'm wrong?
fbsd-kam# /usr/local/sbin/kamailio -V version: kamailio 3.1.5 (i386/freebsd) ec672a flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, PKG_SIZE 4MB poll method support: poll, select, kqueue. id: ec672a compiled on 11:02:58 Nov 15 2011 with gcc 4.2.1
FreeBSD fbsd-kam.dprs-consulting.com 8.2-RELEASE-p2 FreeBSD 8.2-RELEASE-p2 #0: Sun Jul 3 17:49:13 GMT 2011 root@fbsd-kam.dprs-consulting.com:/usr/obj/usr/src/sys/GENERIC i386
Routing script could be sent on request.
Thank you very much!
WBR Dimon
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
On Nov 15, 2011 at 14:43, Dmitry Petrakoff dimon@dprs-consulting.com wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
[...]
10(36629) ERROR: <core> [tcp_main.c:2754]: 10.198.3.100:5060: connect & send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG: <core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Could you try the attached patch and report back if it works? (I couldn't reproduce it on my ancient FreeBSD 7.1)
I think the problem is that newer FreeBSD versions return ENOTCONN when attempting to send on a not yet fully connected socket (instead of EAGAIN).
Andrei [...]
Hi Andrei, hi list!
Of course! Will try it tomorrow morning and will report accordingly!
On 22.11.11 13:07, Andrei Pelinescu-Onciul wrote:
On Nov 15, 2011 at 14:43, Dmitry Petrakoff dimon@dprs-consulting.com wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
[...]
10(36629) ERROR: <core> [tcp_main.c:2754]: 10.198.3.100:5060: connect & send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG: <core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Could you try the attached patch and report back if it works? (I couldn't reproduce it on my ancient FreeBSD 7.1)
I think the problem is that newer FreeBSD versions return ENOTCONN when attempting to send on a not yet fully connected socket (instead of EAGAIN).
Andrei [...]
Hi Andrei!
As promised, I recompiled kamailio with your patch applied.
First quick tests shows that error has gone. Server now able to initiate TCP session and communicate with remote.
Will see how it'll behave under load ( after 14:00 CET ). I will inform you if we'll face with stability problems.
Anyway, thank you very much for quick fix on this problem!
WBR Dimon
On 22.11.11 13:07, Andrei Pelinescu-Onciul wrote:
On Nov 15, 2011 at 14:43, Dmitry Petrakoff dimon@dprs-consulting.com wrote:
Hi List!
I'm running Kamailio for about 1 year without any serious problems. When it works with UDP only, it runs perfectly well for us. Without any major issues.
But now we have a customer who wants to connect Lync to our network. Unfortunately, Lync does not support UDP for SIP signalling, so we tried to teach kamailio to send SIP messages via TCP. And getting strange error:
[...]
10(36629) ERROR: <core> [tcp_main.c:2754]: 10.198.3.100:5060: connect & send for 0x28d988ac failed: Socket is not connected (57) 10(36629) DEBUG: <core> [tcp_main.c:2955]: tcpconn_chld_put: destroying connection 0x28d988ac (1, -1) flags 0060 10(36629) ERROR: tm [../../forward.h:170]: msg_send: ERROR: tcp_send failed
Could you try the attached patch and report back if it works? (I couldn't reproduce it on my ancient FreeBSD 7.1)
I think the problem is that newer FreeBSD versions return ENOTCONN when attempting to send on a not yet fully connected socket (instead of EAGAIN).
Andrei [...]