Re: [Kamailio-Users] location table

Hello,

On 02/05/2009 10:06 PM, joy yue wrote:

Hi Daniel,

Yes, registration is authenticated against db. Actually, other transactions like INVITE also are authenticated against the db. But when things are not going well, the yield() function call is from registration scenario.

start the openser with debug=5, search after lines containing "locks array size" and send them. Maybe the number of the locks that could be created for usrloc is low.

Thanks, Daniel

Here is the callstack:

          libc.so.1`sched_yield
          usrloc.so`new_ucontact+0x470
          usrloc.so`insert_ucontact+0x28
          registrar.so`save+0x674
          openser`do_action+0x2e74
          openser`run_action_list+0x5c
          openser`eval_expr+0xaf8
          openser`eval_expr+0x2950
          openser`eval_expr+0x2980
          openser`do_action+0x24d0
          openser`run_action_list+0x5c
          openser`do_action+0x1738
          openser`run_action_list+0x5c
          openser`do_action+0x2628
          openser`run_top_route+0x25c
          openser`receive_msg+0x6ec
          openser`udp_rcv_loop+0x1740
          openser`main+0x3f9c
          openser`_start+0x7c

-Joy

On Thu, Feb 5, 2009 at 11:57 AM, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:

Hello,


On 02/05/2009 09:48 PM, joy yue wrote:


    Hi Daniel,

    db_mode is 0. I am using 1.3.2 version. You mentioned there is
    compilation option to disable syslog, how can I do that?

check the Makefile.defs, you have to enable -DNO_LOG in DEFS variable.

Are you doing also authentication of REGISTERs against db?

Cheers,
Daniel



    thanks,
    -Joy


    On Thu, Feb 5, 2009 at 10:59 AM, Daniel-Constantin Mierla
    <miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote:

       Hello Joy,


       On 02/05/2009 08:41 PM, joy yue wrote:


           Hi Daniel,

           I checked the NW traffic, it's not very high, around 11k
           pkt/sec coming into SUT. With the similar amount of
    traffic,
           openser has not problem to deal with call
    setup/teardown, but
           not registration. Increasing udp buffer helps some, but
    not a lot.


       then probably is whether syslog or db connection if you do
       authentication.

       If CSeq is not properly incremented by uac and openser/kamailio
       starts complaining a lot to a syncronous syslog, then it is
    very
       likely to be the cause. Later versions of kamailio should
    support
       negative values for debug, try to set it to -3.

       Also, what is the db_mode parameter value of usrloc?

       Cheers,
       Daniel


           I am using 32 children. When things get wrong, with the
    error
           message I mentioned earlier, I can see messages
    generated by
           syslog can be very large.  This may further degrade the
           performance and more packets get dropped.

           How much rate do you get for registration? Do you use
           authentication?

           I will try the test with larger hash size.

           Thanks,
           -Joy

           On Thu, Feb 5, 2009 at 10:20 AM, Daniel-Constantin Mierla
           <miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>
           <mailto:miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>> wrote:

              Hello Joy,


              On 02/05/2009 08:07 PM, joy yue wrote:


                  Hi Daniel,



                  On Thu, Feb 5, 2009 at 7:29 AM,
    Daniel-Constantin Mierla
                  <miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>
           <mailto:miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>
                  <mailto:miconda@gmail.com
    <mailto:miconda@gmail.com> <mailto:miconda@gmail.com
    <mailto:miconda@gmail.com>>
           <mailto:miconda@gmail.com <mailto:miconda@gmail.com>
    <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>>>> wrote:

                     Hello,

                     are you running kamailio/openser with higher
    debug
           mode (e.g.,
                     debug set to a value higher than 3)?


                  The debug mode I am using is 0, which is set in the
           .cfg file.

              ok, then is not high at all :-)



                     Is your syslog configured asynchronous? I tested
                  registration with
                     very high rate and there was no performance
    issue.
           There is for
                     sure something misconfigured.


                  I am using solaris. The default mode is
    synchronous, and
                  actually I don't think solaris support
    asynchronous write.
                  openser use syslog to log in debugging
    information, right?


              by default yes. Do you get lot of log messages
    coming from
           openser
              in the syslog file?


                  If the mode matters, can I just simply disable
    openser from
                  logging in any information to files?


              There is compilation to disable all loggings, but it
    seems that
              the problem is somewhere else.

              What is the value of children in your configuration
    file? Also,
              try to increase the size of the hash table used by
    usrloc
           module:

    http://kamailio.org/docs/modules/1.4.x/usrloc.html#id2506464

              Can you get stats from the network and see what was the
           reason for
              dropped packets? Full buffer?

              Cheers,
              Daniel



                  thanks,
                  -Joy

                     Cheers,
                     Daniel



                     On 02/05/2009 01:37 AM, joy yue wrote:


                         Hi Henning/Daniel,

                         Sorry to take such a long time replying back.

                         For my rig, I am using the memory as location
           back-up. When
                         the issue occurs, I see many registration
           request sent from
                         SIPp but not many 200 replies. netstat
    shows a huge
                  amount of
                         packets get dropped.

                         Also I realized the issue only occurs
    when SIPp
           tried to
                         register many users in a very short time.
    With
           the same
                  number
                         of users, the issue goes away if registration
           rate is kept
                         lower. When the issue occurs, usrloc module
           contends lock a
                         lot calling from new_ucontact(), and many
    system
           time
                  is spent
                         in yield system calls. So it looks more
    like a
           performance
                         issue to me.

                         Thanks,
                         -Joy

                         On 1/27/09, *Henning Westerholt*
                  <henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>>
                         <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>
                  <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>>>
                         <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>
                  <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>>
                         <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>
                  <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>
           <mailto:henning.westerholt@1und1.de
    <mailto:henning.westerholt@1und1.de>>>>>> wrote:

                            On Monday 26 January 2009, joy yue wrote:
                            > Is there a limitation on the number
    of rows in
                  location
                         table?
                            In my rig,
                            > whatever number of users I use
    (>2million
           users), I
                         notice the
                            number of
                            > users in location table is 343707,
    which is far
                  less than the
                            number of
                            > users in my test. I thought
    previously that
           location
                         table saves
                            all the
                            > users in my test.
                            >
                            > Also when I use large number of users
           (>2million),
                         openser pops
                            up an
                            > error: ERROR: registrar:update_contacts:
           invalid
                  cseq for aor
                            <xxxx>. Has
                            > anyone saw this before? I am using
           openser1.3.2.


                            Hi joy,

                            no, there is no such a limitiation,
    we've more
                  registered
                         users.
                            The invalid
                            CSEQ error you see is not related to this
                  observation. The
                         error
                            means that a
                            device tried to do re-registration (same
           callid), but
                         without properly
                            increasing the Cseq number in the REGISTER
           request -
                         RFC3261 says that
                            requests from the same dialog (like
    REGISTER +
                  re-REGISTER)
                         must have
                            increasing cseq.

                            What db_mode do you use in your usrloc? Do
           you see
                  any other
                            errors in the
                            logs?

                            Cheers,


                            Henning



     ------------------------------------------------------------------------




     _______________________________________________
                         Kamailio (OpenSER) - Users mailing list
                         Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>
                  <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>>
                  <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>
                  <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>>>


     http://lists.kamailio.org/cgi-bin/mailman/listinfo/users

     http://lists.openser-project.org/cgi-bin/mailman/listinfo/users


                     --    Daniel-Constantin Mierla
                     http://www.asipto.com



    ------------------------------------------------------------------------

                  _______________________________________________
                  Kamailio (OpenSER) - Users mailing list
                  Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
           <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>>

    http://lists.kamailio.org/cgi-bin/mailman/listinfo/users

    http://lists.openser-project.org/cgi-bin/mailman/listinfo/users


              --    Daniel-Constantin Mierla
              http://www.asipto.com



     ------------------------------------------------------------------------

           _______________________________________________
           Kamailio (OpenSER) - Users mailing list
           Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>
    <mailto:Users@lists.kamailio.org
    <mailto:Users@lists.kamailio.org>>
           http://lists.kamailio.org/cgi-bin/mailman/listinfo/users

     http://lists.openser-project.org/cgi-bin/mailman/listinfo/users


       --    Daniel-Constantin Mierla
       http://www.asipto.com


    ------------------------------------------------------------------------

    _______________________________________________
    Kamailio (OpenSER) - Users mailing list
    Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>
    http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
    http://lists.openser-project.org/cgi-bin/mailman/listinfo/users


-- 
Daniel-Constantin Mierla
http://www.asipto.com

---

---

Kamailio (OpenSER) - Users mailing list Users@lists.kamailio.org http://lists.kamailio.org/cgi-bin/mailman/listinfo/users http://lists.openser-project.org/cgi-bin/mailman/listinfo/users