Hello, I relaxed that condition to not connect on forwarding only for initial requests going though nat. Can you test with latest master and see how is going for your use case? Cheers, Daniel On 09.09.19 02:00, Anthony Joseph Messina wrote: > In preparation for the 5.3 release, I've been testing the following > configuration change for TCP/TLS connections: > > https://github.com/kamailio/kamailio/commit/ > 8bba208fe6ae7ccb4c92362b8c33f1530b9f56da > > route[REQINIT] { > > # no connect for sending replies > set_reply_no_connect(); > if(has_totag()) { > > # no connect for requests within dialog > set_forward_no_connect(); > > } > > This change creates issues when a UAC TLS INVITE routes to an upstream > gateway using TLS to port 5061 (via the LCR module). Kamailio sends the > initial outbound TLS connection from a local ephemeral port. The TCPOPS > tcp_keepalive_enable function issues keepalives from the local ephemeral > port to the gateway port 5061: > > https://kamailio.org/docs/modules/stable/modules/ > tcpops#tcpops.f.tcp_keepalive_enable > > Even so, the TLS connection eventually times out, after which in-dialog > requests from the UAC are no longer able to reach the upstream gateway. > > ERROR: tm [../../core/forward.h:293]: msg_send_buffer(): tcp_send failed > WARNING: tm [t_fwd.c:1570]: t_send_branch(): sending request on branch 0 > failed > ERROR: sl [sl_funcs.c:372]: sl_reply_error(): stateless error reply used: > Unfortunately error on sending to next hop occurred (477/SL) > > I figure I must be doing something wrong with my TCPOPS here. Is a TLS > connection to an upstream gateway supposed to be maintained throughout the > duration of a call?

It no longer looks like an issue related to set_reply_no_connect() or set_forward_no_connect() added by the commit you referenced. Those were added to prevent attempting to connect to devices behind the nat (in that case the device has to maintain the connection, otherwise nobody can connect back to it) as well as prevent someone in the wild sending a request then closing the connection, without waiting for the reply, which is typically routed back to via, commonly with an ephemeral port. The follow up commit I did it in master recently is no longer setting the flag for requests within dialog, but I understand you have connection problems for requests within dialog, am I right? Cheers, Daniel On 10.09.19 01:08, Anthony Joseph Messina wrote: > I still ran into some trouble when one side was NAT'd. > > Am I correct in thinking that it would be undesirable to maintain a > TCP/TLS > connection to an upstream TLS gateway that is using the well-known port > 5061? > > I was thinking this may be a case for TCP_REUSEPORT and force_send_socket, > but that seems a little complex seeing as I can just let Kamailio > reconnect (when necessary) rather than preventing the outbound TLS from > connecting when it would otherwise succeed. > > I'll try and work through more detailed configuration scenarios. -A > > On Monday, September 9, 2019 2:12:07 AM CDT Daniel-Constantin Mierla

>> Hello, >> >> I relaxed that condition to not connect on forwarding only for initial >> requests going though nat. Can you test with latest master and see how >> is going for your use case? >> >> Cheers, >> Daniel >> >> On 09.09.19 02:00, Anthony Joseph Messina wrote: >>> In preparation for the 5.3 release, I've been testing the following >>> configuration change for TCP/TLS connections: >>> >>> https://github.com/kamailio/kamailio/commit/ >>> 8bba208fe6ae7ccb4c92362b8c33f1530b9f56da >>> >>> route[REQINIT] { >>> >>> # no connect for sending replies >>> set_reply_no_connect(); >>> if(has_totag()) { >>> >>> # no connect for requests within dialog >>> set_forward_no_connect(); >>> >>> } >>> >>> This change creates issues when a UAC TLS INVITE routes to an upstream >>> gateway using TLS to port 5061 (via the LCR module). Kamailio sends the >>> initial outbound TLS connection from a local ephemeral port. The TCPOPS >>> tcp_keepalive_enable function issues keepalives from the local ephemeral >>> port to the gateway port 5061: >>> >>> https://kamailio.org/docs/modules/stable/modules/ >>> tcpops#tcpops.f.tcp_keepalive_enable >>> >>> Even so, the TLS connection eventually times out, after which in-dialog >>> requests from the UAC are no longer able to reach the upstream gateway. >>> >>> ERROR: tm [../../core/forward.h:293]: msg_send_buffer(): tcp_send failed >>> WARNING: tm [t_fwd.c:1570]: t_send_branch(): sending request on branch 0 >>> failed >>> ERROR: sl [sl_funcs.c:372]: sl_reply_error(): stateless error reply >>> used: >>> Unfortunately error on sending to next hop occurred (477/SL) >>> >>> I figure I must be doing something wrong with my TCPOPS here. Is a TLS >>> connection to an upstream gateway supposed to be maintained throughout >>> the >>> duration of a call?

On 15.09.19 03:34, Anthony Joseph Messina wrote: You have to differentiate between calls with one side behind nat and the other one on a pubic IP that is like a server/gateway and can accept new connections, even for requests within dialog. My initial change to the default config file was done in the perspective that the respective configuration is routing between local users, where is not common for a user device to register, then close the connection, because it was using a ephemeral port anyhow. Cheers, Daniel > On Wednesday, September 11, 2019 1:21:34 AM CDT Daniel-Constantin Mierla > > wrote: >> It no longer looks like an issue related to set_reply_no_connect() or >> set_forward_no_connect() added by the commit you referenced. Those were >> added to prevent attempting to connect to devices behind the nat (in >> that case the device has to maintain the connection, otherwise nobody >> can connect back to it) as well as prevent someone in the wild sending a >> request then closing the connection, without waiting for the reply, >> which is typically routed back to via, commonly with an ephemeral port. >> >> The follow up commit I did it in master recently is no longer setting >> the flag for requests within dialog, but I understand you have >> connection problems for requests within dialog, am I right? >> >> Cheers, >> Daniel >> >> On 10.09.19 01:08, Anthony Joseph Messina wrote: >>> I still ran into some trouble when one side was NAT'd. >>> >>> Am I correct in thinking that it would be undesirable to maintain a >>> TCP/TLS >>> connection to an upstream TLS gateway that is using the well-known port >>> 5061? >>> >>> I was thinking this may be a case for TCP_REUSEPORT and >>> force_send_socket, >>> but that seems a little complex seeing as I can just let Kamailio >>> reconnect (when necessary) rather than preventing the outbound TLS from >>> connecting when it would otherwise succeed. >>> >>> I'll try and work through more detailed configuration scenarios. -A >>> >>> On Monday, September 9, 2019 2:12:07 AM CDT Daniel-Constantin Mierla > > wrote: >>>> Hello, >>>> >>>> I relaxed that condition to not connect on forwarding only for initial >>>> requests going though nat. Can you test with latest master and see how >>>> is going for your use case? >>>> >>>> Cheers, >>>> Daniel >>>> >>>> On 09.09.19 02:00, Anthony Joseph Messina wrote: >>>>> In preparation for the 5.3 release, I've been testing the following >>>>> configuration change for TCP/TLS connections: >>>>> >>>>> https://github.com/kamailio/kamailio/commit/ >>>>> 8bba208fe6ae7ccb4c92362b8c33f1530b9f56da >>>>> >>>>> route[REQINIT] { >>>>> >>>>> # no connect for sending replies >>>>> set_reply_no_connect(); >>>>> if(has_totag()) { >>>>> >>>>> # no connect for requests within dialog >>>>> set_forward_no_connect(); >>>>> >>>>> } >>>>> >>>>> This change creates issues when a UAC TLS INVITE routes to an upstream >>>>> gateway using TLS to port 5061 (via the LCR module). Kamailio sends >>>>> the >>>>> initial outbound TLS connection from a local ephemeral port. The >>>>> TCPOPS >>>>> tcp_keepalive_enable function issues keepalives from the local >>>>> ephemeral >>>>> port to the gateway port 5061: >>>>> >>>>> https://kamailio.org/docs/modules/stable/modules/ >>>>> tcpops#tcpops.f.tcp_keepalive_enable >>>>> >>>>> Even so, the TLS connection eventually times out, after which >>>>> in-dialog >>>>> requests from the UAC are no longer able to reach the upstream >>>>> gateway. >>>>> >>>>> ERROR: tm [../../core/forward.h:293]: msg_send_buffer(): tcp_send >>>>> failed >>>>> WARNING: tm [t_fwd.c:1570]: t_send_branch(): sending request on branch >>>>> 0 >>>>> failed >>>>> ERROR: sl [sl_funcs.c:372]: sl_reply_error(): stateless error reply >>>>> used: >>>>> Unfortunately error on sending to next hop occurred (477/SL) >>>>> >>>>> I figure I must be doing something wrong with my TCPOPS here. Is a >>>>> TLS >>>>> connection to an upstream gateway supposed to be maintained throughout >>>>> the >>>>> duration of a call?