Hey Guys,
After upgrading from 3.2.x to 3.3.0 (out of debian packages and fixing db side) Kamailio is crashing on processing REGISTER. This crash only happens for updates (or if a stale contact for the same AOR is already present), not initial REGISTERs.
Attached you can find the log taken with debug=6 as well as the REGISTER package producing the segfault.
Kamailio version: """ root@PrxyDev:/etc/slocator# kamailio -V version: kamailio 3.3.0 (i386/linux) flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled on 22:17:07 Jun 18 2012 with gcc 4.4.5 """
Syslog msg in debug=2: """ Jun 27 15:46:22 PrxyDev kernel: [11330.002854] kamailio[6959]: segfault at 0 ip (null) sp bf8d531c error 4 in kamailio[8048000+1ea000] Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6970]: : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 10 Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6957]: ALERT: <core> [main.c:785]: child process 6959 exited by a signal 11 Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6957]: ALERT: <core> [main.c:788]: core was not generated """
Thanks in advance for any tip! DanB
Hello,
if you can reproduce it, try to get the core dump, some steps that will help getting one easier:
- run as root - before starting do: 'ulimit -c unlimited'
Retest in order to get the issue again and if you get the corefile, then send the backtrace:
gdb /path/to/kamailio /path/to/corefile bt full
Cheers, Daniel
On 6/27/12 4:01 PM, Dan-Cristian Bogos wrote:
Hey Guys,
After upgrading from 3.2.x to 3.3.0 (out of debian packages and fixing db side) Kamailio is crashing on processing REGISTER. This crash only happens for updates (or if a stale contact for the same AOR is already present), not initial REGISTERs.
Attached you can find the log taken with debug=6 as well as the REGISTER package producing the segfault.
Kamailio version: """ root@PrxyDev:/etc/slocator# kamailio -V version: kamailio 3.3.0 (i386/linux) flags: STATS: Off, USE_IPV6, USE_TCP, USE_TLS, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, F_MALLOC, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 4MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: unknown compiled on 22:17:07 Jun 18 2012 with gcc 4.4.5 """
Syslog msg in debug=2: """ Jun 27 15:46:22 PrxyDev kernel: [11330.002854] kamailio[6959]: segfault at 0 ip (null) sp bf8d531c error 4 in kamailio[8048000+1ea000] Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6970]: : <core> [pass_fd.c:293]: ERROR: receive_fd: EOF on 10 Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6957]: ALERT: <core> [main.c:785]: child process 6959 exited by a signal 11 Jun 27 15:46:22 PrxyDev /usr/sbin/kamailio[6957]: ALERT: <core> [main.c:788]: core was not generated """
Thanks in advance for any tip! DanB
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Hey Daniel,
Thanks for getting to me back so fast.
Attached you can have the trace you asked. Feel free to come back for any tests necessary to spot the issue.
BR, DanB
Hello,
the frame 0 (where the crash happened) is completely empty, something went wrong there. Did you install from sources or packages?
Set debug=3 and send me all the logs for processing the REGISTER that causes the issue. Also, send again the backtrace from the new corefile, maybe this time will be complete.
Cheers, Daniel
On 6/28/12 1:55 PM, Dan-Cristian Bogos wrote:
Hey Daniel,
Thanks for getting to me back so fast.
Attached you can have the trace you asked. Feel free to come back for any tests necessary to spot the issue.
BR, DanB
Daniel,
Attaching requested files. This time I got two core files, analyzed them separately with gdb.
Regarding install sources, I did it via debian packages promoted on the website (kamailio33 squeeze).
Thanks, DanB
Hello,
still empty content for frame 0 in both backtraces, but I figure out the issue from the log messages.
p_usrloc module is exporting usrloc module API to be used by registrar module, but for 3.3.0, usrloc got new API members to handle SIP GRUU and Outbound extensions. They were not implemented for p_usrloc, so the crash is because accessing some functions which don't exists. That makes p_usrloc unusable with 3.3 branch by now. It will take a bit to do a fix for it, so if you need p_usrloc, you have to use 3.2.x at this moment.
Cheers, Daniel
On 6/28/12 3:15 PM, Dan-Cristian Bogos wrote:
Daniel,
Attaching requested files. This time I got two core files, analyzed them separately with gdb.
Regarding install sources, I did it via debian packages promoted on the website (kamailio33 squeeze).
Thanks, DanB
Hello,
we know about this and we will extend the p_usrloc Module in the near future. Do you consider this is a bug related to the p_usrloc (it's unusable at the moment) or as a feature enhancement? I've talked with Marius and he thought initially that this is an enhancement and that's why we haven't put any effort in this. On the other hand, if it's a bug and the module is not usable, we will give it higher priority and will fix this as soon as possible.
Kind Regards, Paul
we know about this bug and we are going to Am 28.06.2012 16:23, schrieb Daniel-Constantin Mierla:
Hello,
still empty content for frame 0 in both backtraces, but I figure out the issue from the log messages.
p_usrloc module is exporting usrloc module API to be used by registrar module, but for 3.3.0, usrloc got new API members to handle SIP GRUU and Outbound extensions. They were not implemented for p_usrloc, so the crash is because accessing some functions which don't exists. That makes p_usrloc unusable with 3.3 branch by now. It will take a bit to do a fix for it, so if you need p_usrloc, you have to use 3.2.x at this moment.
Cheers, Daniel
On 6/28/12 3:15 PM, Dan-Cristian Bogos wrote:
Daniel,
Attaching requested files. This time I got two core files, analyzed them separately with gdb.
Regarding install sources, I did it via debian packages promoted on the website (kamailio33 squeeze).
Thanks, DanB
Hello,
On 6/28/12 5:11 PM, Pawel Kuzak wrote:
Hello,
we know about this and we will extend the p_usrloc Module in the near future. Do you consider this is a bug related to the p_usrloc (it's unusable at the moment) or as a feature enhancement? I've talked with Marius and he thought initially that this is an enhancement and that's why we haven't put any effort in this. On the other hand, if it's a bug and the module is not usable, we will give it higher priority and will fix this as soon as possible.
it is a bug IMO, it should not crash at least -- my idea is to just ignore GRUU/Outbound extensions if it requires lot of coding just for them, so eventually it will work like before. But if you can make a patch for it, should be applied asap to get it back working.
Cheers, Daniel
Kind Regards, Paul
we know about this bug and we are going to Am 28.06.2012 16:23, schrieb Daniel-Constantin Mierla:
Hello,
still empty content for frame 0 in both backtraces, but I figure out the issue from the log messages.
p_usrloc module is exporting usrloc module API to be used by registrar module, but for 3.3.0, usrloc got new API members to handle SIP GRUU and Outbound extensions. They were not implemented for p_usrloc, so the crash is because accessing some functions which don't exists. That makes p_usrloc unusable with 3.3 branch by now. It will take a bit to do a fix for it, so if you need p_usrloc, you have to use 3.2.x at this moment.
Cheers, Daniel
On 6/28/12 3:15 PM, Dan-Cristian Bogos wrote:
Daniel,
Attaching requested files. This time I got two core files, analyzed them separately with gdb.
Regarding install sources, I did it via debian packages promoted on the website (kamailio33 squeeze).
Thanks, DanB
-
Dan-Cristian Bogos -
Daniel-Constantin Mierla -
Pawel Kuzak