Yikes, I don't have pcaps of this issue anymore and it is difficult to reproduce because it involves a random collision of port numbers on the customer's network with NAT involved. On top of that I seem to be having trouble getting my issue across.
In this case I have two connections created by phones with the same destination but different sources, then a third which has a the same source but different destination. Device A: tcp:CustomerIP:10042 -> `tcp:KamailioIP:4242` Device B: `tcp:CustomerIP:11042` -> `tcp:KamailioIP:4242` Device A new connection for a 200 OK response: `tcp:CustomerIP:11042` -> tcp:KamailioIP:5062
Kamailio sees this as: Device A: `tcp:KamailioIP:4242` -> tcp:CustomerIP:10042 Device B: `tcp:KamailioIP:4242` -> `tcp:CustomerIP:11042` Device A receives a SIP message with a contact telling them to respond to Kamailio on 5062, which causes it to create a new TCP connection which in turn confuses Kamailio. Device A tcp:KamailioIP:5062 -> `tcp:CustomerIP:11042`
Kamailio is reusing the tcp connection whenever the destination address is matching an active one.
As seen above, Kamailio thinks Device A has two connections with the same destination. This invite is not in response to any packet, it is being relayed from my PBX as the first packet of a new transaction and I am using lookup() and then t_relay() to set the Request-URI and then relay the packet to that destination. If t_relay uses the R-URI destination to find the TCP socket it should send from, how does it choose between these two sockets? The problem I am seeing is that it incorrectly chooses to send the INVITE from the `socket' that has a source port of 5062 because the newest connection it has become aware of for that destination is the socket with 5062.
I hope I've been able to clearly describe what, I think, is happening here. If not, we currently have a mitigation strategy to avoid sending a packet with a contact that makes phones want to create another connection to Kamailio.