Hi Sharath

Thanks for the tip! In fact just last night I've found what got things stuck was nothing to do with Kamailio - it was one of our own little module (plugged into Kamailio) that's got locked up.

The first thing I suspected was like what you mentioned that Kamailio got stuck due to fragmentation. I wonder if you've seen this happening with Kamailio? At least in this problem of ours it wasn't Kamailio.

Thank you and Daniel for the help on this!

Yufei

On 22 April 2015 at 22:19, Sharath Kumar <Sharath.Kumar@mezocliq.com> wrote:

It is definitely possible due to fragmentation​ the sip receiver was stuck in some weird state. If I were you I would do TCP for SIP. It is also recommended in 3261 to use TCP when sip is that big.


From: sr-users <sr-users-bounces@lists.sip-router.org> on behalf of Yufei Tao <yufei.tao@gmail.com>
Sent: Friday, April 3, 2015 7:04 AM
To: sr-users@lists.sip-router.org

Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU
 
Hi Daniel

We don't have pike module enabled.

When the problem occurred on an VIP, we observed these:
* Kamailio stopped responding to any messages that were sent to the VIP, not just OPTIONS
* The OPTIONS messages are not big. But the other SIP messages, e.g. some of the INVITE/OK that came from some SBCs can be big
* netstat showed the Recv-Q on that VIP had a lot of bytes accumulated, while Kamailio was not seeing/reading them
* Kamailio responded fine on its real IP, when I sent OPTIONS pings to it using sipsak
* After restarting Kamailio it started working. But after 1 week or so the same problem happened again

Since this only happened after running for 1 week or so, we didn't have any traces to show what exactly happened at the particular time when it happened. It is possible some SIP messages may have come in fragmented and some are just too big, depending on the route they came from etc. So I was wonder if it was possible that the UDP receive buffer was filled somehow with messed-up messages. Is there anyway to check this? Any suggestions where I should start looking please? Or is it generally a bad idea to use UDP when there are messages that may be too big, either fragmented or not?

Since the it is running in the production environment, I'd like to get some confidence that a Kamailio upgrade will fix the problem first before I change anything there.

Cheers,
Yufei



Date: Wed, 01 Apr 2015 16:27:44 +0200
From: Daniel-Constantin Mierla <miconda@gmail.com>
To: "Kamailio (SER) - Users Mailing List"
        <sr-users@lists.sip-router.org
>
Subject: Re: [SR-Users] SIP messages over UDP with sizes over MTU
Message-ID: <551C0060.3010002@gmail.com>
Content-Type: text/plain; charset="windows-1252"

Hello,

first, not related to the topic you reported, I would recommend at least
upgrading to latest 4.0.x, there were many fixes from v4.0.0 and there
are no changes to be done to config or database -- simply deploy latest
4.0.x and restart.

Back to this topic. Is all the other traffic handled apart of big OPTIONS?

Do you have pike module enabled? If yes, can you double check and be
sure that the SBC is not blacklisted (traffic from it should not be
handled via pike_check_req()).

Cheers,
Daniel

On 01/04/15 15:42, Yufei Tao wrote:
> Hi
>
> We've got Kamailio (v4.0.0) connected to some SBC, which sends SIP
> traffic and periodic OPTIONS pings to Kamailio's VIP. Kamailio
> responds to the OPTIONS pings with OK, i.e. in the  main route block:
>    if (is_method("OPTIONS"))
>    {
>       sl_send_reply("200","OK");
>       exit;
>    }
>
> All works fine for a week or so, then Kamailio stopped responding to
> the OPTIONS pings on the VIP it listens to. But it still respond to
> OPTIONS pings that are sent to its real IP. The real IP is not used
> for receiving/sending any traffic while only the VIP is. So it seems
> that Kamailio is still working, but maybe having problems with the
> receiving buffer for the VIP?
>
> We do see that some SIP messages sent to Kamailio's VIP are too big
> (sometimes over 1500 bytes). My question is, in this case, what would
> be expected to happen? Is it possible somehow the receiving buffer for
> the VIP got messed up by the big UDP messages? Any one seen similar
> problems? What is the suggested solution?
>
> We're considering moving to TCP. But since this is production
> environment, I want to get some confidence that the problem we saw was
> likely to have been caused by the UDP message being too large.
>
> Cheers,
> Yufei