Re: [SR-Users] Kamailio stability/timing problem w.r.t. registrations?

Hi All, We have solved this issue now. The problem was the wrong interpretation of the Kamailio Makefile.defs makefile; The ARCH script variable was set to a wrong string by our higher level makefile. Ea; for powerPC, Kamilio.defs expects 'ppc' as the value while the higher level makefile sets the ARCH variable to 'powerpc'. So Kamailio was compiled with wrong and missing GCC arguments. Also some C-defines were not set correctly which resulted in linking wrong 'locking' routines, like atomic_unknown.h. Instead atomic_ppc.h should be linked/compiled. Thanks for your support anyway. Best regards, Orhan Yilmaz On Fri, Nov 29, 2013 at 3:02 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com <mailto:miconda@gmail.com>> wrote: Attaching with gdb and getting the backtrace (as Ovidiu said), will be very useful The function futex_wait_queue_me() is from kernel, so it might be an operation from the kernel about reading the udp packages. So we cannot tell without the full trace if a kamailio function is blocking the futex or is other standard library function. I wanted to mention also that poll methods are for tcp only, for udp kamailio uses recvfrom(). Can you also provide the output for 'kamailio -I' (that's upper case i)? Cheers, Daniel On 11/27/13 4:44 PM, Ovidiu Sas wrote: Try to attach gdb to the kamailio processes and run a full backtrace. Regards, Ovidiu Sas On Wed, Nov 27, 2013 at 5:16 AM, Sotas Development &lt;sotasdev(a)gmail.com <mailto:sotasdev@gmail.com>> wrote: Hi, In the mean time we have gathered more information on this problem: As given below, kamailio stops grabbing UDP SIP messages (SIP registrations) after running a while on an embedded ARM and PPC platform (which runs linux 2.6.33 kernel). Some times the hangup occures within hours and some times after couple of days running. NETSTAT OUTPUT: root# netstat -pl | grep kam udp 1047968 0 (null):sip (null):* 8416/kamailio raw 0 0 (null):255 (null):* 255 8416/kamailio unix 2 [ ACC ] STREAM LISTENING 755205 8429/kamailio /tmp/kamailio_ctl Kamailio is started with the following options = -m 4 -n 3 -f <cfg> -D Other relevant info: - When Kamailio hangs, I also noticed that the flag "inuse_transactions" has always the value of '1'. Readout with "kamctl monitor". - A simple cat to /proc/<kamailio_pid>/wchan gives us the function: futex_wait_queue_me. - All possible polling methods are used with -W parameter (sigio_rt, poll, select etc) during these tests. Non of these options did solve this problem. I hope the additional info will clarify more. Thanks in advance. Best regards, Orhan Yilmaz On Wed, Nov 13, 2013 at 6:12 PM, Ovidiu Sas &lt;osas(a)voipembedded.com <mailto:osas@voipembedded.com>> wrote: In a previous e-mail, you posted a warning that you had while compiling: "no native memory barrier implementations, falling back to slow lock based workarround" which means that you are already running without atomic locks. Regards, Ovidiu Sas On Wed, Nov 13, 2013 at 10:40 AM, Sotas Development &lt;sotasdev(a)gmail.com <mailto:sotasdev@gmail.com>> wrote: Hi, Here's an update of this topic. We've tried again with the latest stable version 4.0.4. Unfortunately the problem still exists. In mails above it is mentioned to use kamailio without atomic locks. How do we this (e.g. which makefile options)? Kind regards, Bert (on behalf of Michiel Veldkamp) On Mon, Jan 28, 2013 at 4:44 PM, Ovidiu Sas &lt;osas(a)voipembedded.com <mailto:osas@voipembedded.com>> wrote: 4.0 (current trunk) is in code freeze. I would suggest to test the trunk version (next 4.0). Even openser 1.3 requires patches to be properly cross compiled. Regards, Ovidiu Sas -- VoIP Embedded, Inc. http://www.voipembedded.com ---------- Forwarded message ---------- From: Sotas Development &lt;sotasdev(a)gmail.com <mailto:sotasdev@gmail.com>> Date: Mon, Jan 28, 2013 at 10:08 AM Subject: Re: [SR-Users] Kamailio stability/timing problem w.r.t. registrations? To: "SIP Router - Kamailio (OpenSER) and SIP Express Router (SER) - Users Mailing List" &lt;sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org>> Hi Ovidiu, Thanks for the warning! We did not yet have much success running the current master branch, though this may well be a resource problem on the target platform. For the moment, we decided to switch back to openser 1.3.5 and wait for the official 4.0 release. Regards, Michiel Veldkamp On Thu, Jan 17, 2013 at 7:01 PM, Ovidiu Sas &lt;osas(a)voipembedded.com <mailto:osas@voipembedded.com>> wrote: If you are running the stable version, there's need for heavy Makefile patching in order to properly cross compile (not to include and link to host libs). The trunk has everything fixed and it's cross-compiling properly for most of the modules. Make sure that your binaries are properly cross compiled. Depending on your ARM CPU, atomic locks may or may not work. I tested openser without atomic locks (using regular locks) and it worked fine. Regards, Ovidiu Sas -- VoIP Embedded, Inc. http://www.voipembedded.com _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- VoIP Embedded, Inc. http://www.voipembedded.com _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users(a)lists.sip-router.org <mailto:sr-users@lists.sip-router.org> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda