Daniel, I've been trying a litte longer with the server and I captured one ACK that definetely has RPID:

ACK sip:1054111556446967@cc1int.x.com.ar SIP/2.0
Via: SIP/2.0/UDP 192.168.200.11:5060;branch=z9hG4bK31a69f48;rport
From: "541160911100" <sip:541160911100@192.168.200.11>;tag=as799b3334
To: <sip:1054111556446967@cc1int.cpsnet.com.ar>;tag=f8f2ab2c1295e90ed7dbb499b30f44b2.90f4
Contact: <sip:541160911100@192.168.200.11>
Call-ID: 5d7adcce1399bafd45c194b049479a1b@192.168.200.11
CSeq: 102 ACK
User-Agent: Metrotel
Max-Forwards: 70
Remote-Party-ID: "541160911100" <sip:541160911100@192.168.200.11>;privacy=off;screen=no
Content-Length: 0

Hope it helps,
Uriel

On Mon, Feb 23, 2009 at 12:08 PM, Uriel Rozenbaum <uriel.rozenbaum@gmail.com> wrote:
I didn't really capture the signaling for the issue, I'll get a trace when I get the chance and send it to you.

Rgds,
Uriel


On Mon, Feb 23, 2009 at 12:02 PM, Daniel-Constantin Mierla <miconda@gmail.com> wrote:


On 02/23/2009 02:25 PM, Uriel Rozenbaum wrote:
Sure, I can turn of report ack to check; anyway i substracter $re from db_extra and now I'm not seeing any crashes. But maybe its useful for you to clear the bug in there.
yes, this is the goal. This issue has to be fixed. You can keep the report ack on, I just wanted to get your confirmation.

I will try to reproduce first by myself, and then ask you for more. Just in case you have such ACK at your hand, send it to me, I will use it in tests. Does it have a RPID header?

Thanks,
Daniel



Let me know how you want to proceed and I'll send you everithing.

-- Uriel

On Mon, Feb 23, 2009 at 10:18 AM, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote:

   Hello,

   thanks. Do you have report ack set for acc module?

   The crash happens to an ACK and relates to building outgoing
   request and processing lumps. I will investigate and try to
   reproduce. Just asking for now, would be any chance to get the ACK
   and eventually your config in case it is needed for troubleshooting?

   Thanks,
   Daniel



   On 02/20/2009 08:17 PM, Uriel Rozenbaum wrote:

       Here it is:

       [root@ops3 ~]# gdb /usr/local/sbin/kamailio /core.4899
       GNU gdb Red Hat Linux (6.5-37.el5_2.2rh)
       Copyright (C) 2006 Free Software Foundation, Inc.
       GDB is free software, covered by the GNU General Public
       License, and you are
       welcome to change it and/or distribute copies of it under
       certain conditions.
       Type "show copying" to see the conditions.
       There is absolutely no warranty for GDB.  Type "show warranty"
       for details.
       This GDB was configured as "i386-redhat-linux-gnu"...Using
       host libthread_db library "/lib/libthread_db.so.1".


       warning: Can't read pathname for load map: Input/output error.
       Reading symbols from /lib/libdl.so.2...done.
       Loaded symbols for /lib/libdl.so.2
       Reading symbols from /lib/libresolv.so.2...done.
       Loaded symbols for /lib/libresolv.so.2
       Reading symbols from /lib/libc.so.6...done.
       Loaded symbols for /lib/libc.so.6
       Reading symbols from /lib/ld-linux.so.2...done.
       Loaded symbols for /lib/ld-linux.so.2
       Reading symbols from
       /usr/local/lib/kamailio/modules/db_mysql.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/db_mysql.so
       Reading symbols from /usr/lib/mysql/libmysqlclient.so.15...done.
       Loaded symbols for /usr/lib/mysql/libmysqlclient.so.15
       Reading symbols from /usr/lib/libz.so.1...done.
       Loaded symbols for /usr/lib/libz.so.1
       Reading symbols from /lib/libcrypt.so.1...done.
       Loaded symbols for /lib/libcrypt.so.1
       Reading symbols from /lib/libnsl.so.1...done.
       Loaded symbols for /lib/libnsl.so.1
       Reading symbols from /lib/libm.so.6...done.
       Loaded symbols for /lib/libm.so.6
       Reading symbols from /lib/libssl.so.6...done.
       Loaded symbols for /lib/libssl.so.6
       Reading symbols from /lib/libcrypto.so.6...done.
       Loaded symbols for /lib/libcrypto.so.6
       Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
       Loaded symbols for /usr/lib/libgssapi_krb5.so.2
       Reading symbols from /usr/lib/libkrb5.so.3...done.
       Loaded symbols for /usr/lib/libkrb5.so.3
       Reading symbols from /lib/libcom_err.so.2...done.
       Loaded symbols for /lib/libcom_err.so.2
       Reading symbols from /usr/lib/libk5crypto.so.3...done.
       Loaded symbols for /usr/lib/libk5crypto.so.3
       Reading symbols from /usr/lib/libkrb5support.so.0...done.
       Loaded symbols for /usr/lib/libkrb5support.so.0
       Reading symbols from /lib/libkeyutils.so.1...done.
       Loaded symbols for /lib/libkeyutils.so.1
       Reading symbols from /lib/libselinux.so.1...done.
       Loaded symbols for /lib/libselinux.so.1
       Reading symbols from /lib/libsepol.so.1...done.
       Loaded symbols for /lib/libsepol.so.1
       Reading symbols from /usr/local/lib/kamailio/modules/sl.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/sl.so
       Reading symbols from /usr/local/lib/kamailio/modules/tm.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/tm.so
       Reading symbols from /usr/local/lib/kamailio/modules/rr.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/rr.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/maxfwd.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/maxfwd.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/usrloc.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/usrloc.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/registrar.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/registrar.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/textops.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/textops.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/mi_fifo.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/mi_fifo.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/uri_db.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/uri_db.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/uri.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/uri.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/xlog.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/xlog.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/acc.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/acc.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/auth.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/auth.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/auth_db.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/auth_db.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/alias_db.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/alias_db.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/domain.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/domain.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/lcr.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/lcr.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/avpops.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/avpops.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/permissions.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/permissions.so
       Reading symbols from
       /usr/local/lib/kamailio/modules/carrierroute.so...done.
       Loaded symbols for /usr/local/lib/kamailio/modules/carrierroute.so
       Reading symbols from /lib/libnss_files.so.2...done.
       Loaded symbols for /lib/libnss_files.so.2
       Core was generated by `/usr/local/sbin/kamailio -P
       /var/run/kamailio.pid'.
       Program terminated with signal 11, Segmentation fault.
       #0  lumps_len (msg=0x81b3450, lumps=<value optimized out>,
       send_sock=0x81afe28) at msg_translator.c:665
       665       switch(r->op){
       (gdb) bt
       #0  lumps_len (msg=0x81b3450, lumps=<value optimized out>,
       send_sock=0x81afe28) at msg_translator.c:665
       #1  0x0807ce8e in build_req_buf_from_sip_req (msg=0x81b3450,
       returned_len=0xbffd27f8, send_sock=0x81afe28, proto=1,
       flags=<value optimized out>) at msg_translator.c:1456
       #2  0x08063703 in forward_request (msg=0x81b3450, p=0x81b28b8)
       at forward.c:362
       #3  0x00157a28 in t_relay_to (p_msg=0x81b3450,
       proxy=0x81b2dc0, flags=0) at t_funcs.c:238
       #4  0x0016a241 in w_t_relay (p_msg=0x81b3450, proxy=0x0,
       flags=0x0) at tm.c:977
       #5  0x08052f66 in do_action (a=0x81ac3b8, msg=0x81b3450) at
       action.c:850
       #6  0x080557d9 in run_action_list (a=0x81ac3b8, msg=0x81b3450)
       at action.c:138
       #7  0x080a4925 in eval_expr (e=0x81ac420, msg=0x81b3450,
       val=0x0) at route.c:1116
       #8  0x080a43a0 in eval_expr (e=0x81ac448, msg=0x81b3450,
       val=0x0) at route.c:1429
       #9  0x080a4336 in eval_expr (e=0x81ac470, msg=0x81b3450,
       val=0x0) at route.c:1434
       #10 0x08052a47 in do_action (a=0x81ac588, msg=0x81b3450) at
       action.c:705
       #11 0x080557d9 in run_action_list (a=0x81ac588, msg=0x81b3450)
       at action.c:138
       #12 0x08054048 in do_action (a=0x81ab960, msg=0x81b3450) at
       action.c:118
       #13 0x080557d9 in run_action_list (a=0x81ab8f8, msg=0x81b3450)
       at action.c:138
       #14 0x08054485 in do_action (a=0x81ab9c8, msg=0x81b3450) at
       action.c:722
       #15 0x080557d9 in run_action_list (a=0x81aa930, msg=0x81b3450)
       at action.c:138
       #16 0x08055b7c in run_top_route (a=0x81aa930, msg=0x81b3450)
       at action.c:118
       #17 0x08094e29 in receive_msg (
          buf=0x817a0a0 "ACK sip:8888860911167@192.168.200.63
       <mailto:sip%3A8888860911167@192.168.200.63>
       <mailto:sip%3A8888860911167@192.168.200.63
       <mailto:sip%253A8888860911167@192.168.200.63>> SIP/2.0\r\nVia:

       SIP/2.0/UDP
       192.168.200.62:5060;branch=z9hG4bK4859c659;rport\r\nRoute:
       <sip:192.168.200.9;lr=on>\r\nFrom: \"7001\"
       <sip:60911100@ops3.cpsnet.com.ar
       <mailto:sip%3A60911100@ops3.cpsnet.com.ar>
       <mailto:sip%3A60911100@ops3.cpsnet.com.ar
       <mailto:sip%253A60911100@ops3.cpsnet.com.ar>>>;tag=as"...,

       len=523, rcv_info=0xbffd3904) at receive.c:165

       #18 0x080d8027 in udp_rcv_loop () at udp_server.c:449
       #19 0x0806d49e in main (argc=3, argv=0xbffd3af4) at main.c:780

       On Fri, Feb 20, 2009 at 3:00 PM, Daniel-Constantin Mierla
       <miconda@gmail.com <mailto:miconda@gmail.com>
       <mailto:miconda@gmail.com <mailto:miconda@gmail.com>>> wrote:

          First send the backtrace, the core without binaries is not much
          helpful as the symbols can differ to a local compiled
       version. Use:

          gdb /usr/local/sbin/kamailio /path/to/corefile

          The do "bt" there and send the output here.

          Thanks,
          Daniel



          On 02/20/2009 02:12 PM, Uriel Rozenbaum wrote:

              Hi Henning, here's the core file zipped.

              On Fri, Feb 20, 2009 at 7:24 AM, Henning Westerholt
              <henning.westerholt@1und1.de
       <mailto:henning.westerholt@1und1.de>
              <mailto:henning.westerholt@1und1.de
       <mailto:henning.westerholt@1und1.de>>
              <mailto:henning.westerholt@1und1.de
       <mailto:henning.westerholt@1und1.de>
              <mailto:henning.westerholt@1und1.de
       <mailto:henning.westerholt@1und1.de>>>> wrote:

                 On Thursday 19 February 2009, Uriel Rozenbaum wrote:
                 > Sure, but I'll need some guidance on obtaining it;
       I'm just
                 checking with
                 > the syslog messages with debugging level in 3. Can you
              show me
                 how to do
                 > it?

                 Hi Uriel,

                 the core was already generated, as you can see in
       the logs:

                 >>    Feb 19 08:48:58 ops3
       /usr/local/sbin/kamailio[4895]:
                 >>    INFO:core:handle_sigs: core was generated

                 You should find this file in the working directory
       of the
              server.

                 Cheers,

                 Henning



          --    Daniel-Constantin Mierla
          http://www.asipto.com


       ------------------------------------------------------------------------

       _______________________________________________
       Kamailio (OpenSER) - Users mailing list
       Users@lists.kamailio.org <mailto:Users@lists.kamailio.org>

--
Daniel-Constantin Mierla
http://www.asipto.com