Hello,
actually your expectation that packets should come in order is "naive",
just think about how internet is constructed and IP routing works. In
the past it was easy to reproduce on mobile networks scenarios when
sending CANCEL very quickly after the INVITE resulted in CANCEL arriving
first at the proxy, then the INVITE.
Probably you don't get it in your lab environment where you have sipp on
the same system as the sip server or one network segment away, but over
the internet the packets can get in different order because of network
transmission (different IP routing paths). It is the reason in some
cases there are jitter buffers and sequence numbers (e.g., in RTP and
SIP (CSeq)). In other words, the protocols like RTP or SIP were designed
to deal with these out-of-order packets.
Ans here is a wikipedia short article enumerating what can cause out of
order:
-
https://en.wikipedia.org/wiki/Out-of-order_delivery
That said, if you missed in the message from mailing list archive that
you linked to, there is a global parameter that should reduce the risk
of sending out of order sip packets to the minimum that can be done from
SIP processing point of view based on call-id, but there are still
chances that on multi-cpu systems the packets read from the network can
get to be processed in different order because of how read on udp
sockets is done by kernel/libc and how cpu scheduler allocate cycles to
running application processes.
Cheers,
Daniel
On 05.12.22 19:34, Jawaid Bazyar wrote:
Hi,
I have experienced out of order packet processing when testing a
simple Kamailio config.
The configuration is as follows, basically:
request_route{
record_route();
enum_query();
xlog("INVITE ENUM query - To URI $tU");
forward();
}
I saw this thread from 2020:
https://www.mail-archive.com/sr-users@lists.kamailio.org/msg11786.html
<https://www.mail-archive.com/sr-users@lists.kamailio.org/msg11786.html>
The issue seems to be that kamailio process scheduling is naïve –
i.e., incoming SIP packets are processed without regard to whether
packets received before this one, are currently being processed. This
means any packets after the initial INVITE that require more
processing, can get reordered.
In my test lab I have:
SIPp – UAC
Kamailio Proxy
SIPp – UAS
The proxy uses enum NAPR lookups to route calls to +13038151000 to the
UAS.
Now, if I do SIPp UAC o SIPp UAS directly, I have no problems – no out
of order packets.
It is only when I introduce Kamailio in the middle that I get OOO packets.
See the images attached: uac-to-proxy, proxy, and proxy-to-uas.
In this example, Kamailio OOO causes SIPp UAC to fail to “generate
audio” – because UAC does not see packets in the correct order, I
never turns on the simulated audio. Calls that have in-order dialogs,
work fine, and SIPP UAC “pauses” 10 seconds to simulate a phone call.
So far, the error rate runs from 1/1000 to around 1/200 – pretty bad.
In the thread, a few things were suggested.
Have fewer kamailio processes than cores:
Did not resolve issue.
Try route_locks_size = 256
Did not resolve issue. Though, it did alter it
somewhat. But, it is not clear to me how this works – would this
setting restrict the number of open calls to 256?
Have only one kamailio process: (“children=1”)
This works. “Works”, I should say, because this would
greatly restrict total platform scalability to a point where it is
probably useless for my application.
I saw the same issue discussed in the OpenSIPS mailing list from 2010,
and the response was “this is not a bug”.
Well, I respectfully beg to differ with both OpenSIPS and Kamailio –
it IS a bug. I don’t think a proxy should reorder packets involved in
a call in a non-deterministic way. In the Kamailio list thread, Alex
Balashov discusses the contortions he has to go through to avoid
repercussions from this issue.
Kamailio as-is probably works fine for relatively low-volume
operations. And a lot of the feedback is “why are out of order packets
a problem?” OK, sure, in the very specific example given in the 2020
thread, maybe who cares. But in my thinking, there is absolutely
nothing preventing Kamailio from generating much more serious OOO
scenarios that would cause calls to fail. In my example, Kamailio OOO
causes SIPp to fail to “generate audio”. Who knows how other SIP
stacks will behave?
But the more one will try to scale Kamailio, the more significantly
this out of order processing issue will become.
The solution to this seems very simple and straightforward – put
packets to be processed into a queue PER Call-ID, or something along
those lines.
In short, the parallelism should be by call, not by packet.
What say ye? Have I misunderstood something here?
Cheers,
Jawaid
__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
sr-users(a)lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla --
www.asipto.com
www.twitter.com/miconda --
www.linkedin.com/in/miconda