Re: [SR-Users] reload to memory is much slower and has problms after upgrade to 3.3.2 and using mem_join=1

Hi, I recompiled with MEMDBG=1 and installed. here are the results for reloading 5 million rows with MTREE: mem_join=1 -->takes 56 seconds and the real_used_size of shmem is around 1.2Gb. mem_join=0 --> takes 10 seconds and the real_used_size of shmem is around 2Gb does it seems normal? 56 seconds is a lot of time...... by the way, when the f_malloc was used, the size of the real_used shmem was twice smaller. Thanks. On Tue, Nov 20, 2012 at 9:45 PM, Daniel-Constantin Mierla &lt;miconda(a)gmail.com > Hello, > > > On 11/20/12 7:34 PM, Uri Shacked wrote: > > Hi, > > can you be a litle more specific of the steps of the install and where do > i make the changes? > > > in the source tree, edit the file Makefile.defs and set: > > MEMDBG=1 > > then run: > > make all > make install > > > some words of what is the diff between f_malloc and q_malloc will be > great :-). > > > q_malloc is more debugging purposes, keeping more information for each > chunk, therefore the overhead is a bit higher than with f_malloc, but > because keeps more details, it is faster to find the fragments that can be > joined. > > Cheers, > Daniel > > > > thanks, > Uri > > On Tue, Nov 20, 2012 at 6:26 PM, Daniel-Constantin Mierla < > miconda(a)gmail.com> >> Hello, >> >> ok, I will look over it. At this moment the f_malloc (which is enabled >> for 3.3) has a pretty inefficient mem join implementation, can you try with >> q_malloc? Edit Makefile.defs and set: >> >> MEMDBG=1 >> >> Then compile and install. >> >> The join operation should be faster, let's see if you get blocking issues >> with this one. >> >> Cheers, >> Daniel >> >> >> On 11/20/12 2:57 PM, Uri Shacked wrote: >> >> Daniel hi, >> >> I attached 2 txt files. >> One with mem_join=1, the other with mem_join=0, and the info you asked >> for. >> Let me know if it is OK. >> >> Thanks, >> Uri >> >> On Mon, Nov 19, 2012 at 10:50 AM, Daniel-Constantin Mierla < >> miconda(a)gmail.com>> >>> Hello, >>> >>> if you set memjoin to 0, do you see any difference? >>> >>> Can you try again (with memjoin 1 as well as 0) and send the output of: >>> >>> kamctl mi get_statistics shmem: >>> >>> before executing the reload commands? >>> >>> When it gets to 100%, can you see which process is using the cpu and >>> attach to it with: >>> >>> gdb /path/to/kamailio PID >>> >>> then do: >>> >>> bt full >>> >>> and send output here? >>> >>> Cheers, >>> Daniel >>> >>> >>> On 11/18/12 4:09 PM, Uri Shacked wrote: >>> >>> After some testing I notice the following: >>> First reload of 5 million records after kamailio started took about 9 >>> sec. >>> Second reload (4 minutes after the first one) took 60 sec. >>> The third one (again about 4 minutes after the secind) got kamailio to >>> use 100% cpu and after 13 minutes! i killed it..... >>> >>> I can understand that the memory manger works harder, still, any ideas >>> on how to use mem_join and keep on reloading data. >>> (in real life our data loads 5 million records once a day when almost no >>> traffic. still after a few days it stops...) >>> >>> Thanks, >>> Uri >>> >>> >>> >>> On Sun, Nov 18, 2012 at 11:52 AM, Uri Shacked &lt;ushacked(a)gmail.com&gt;wrote;wrote: >>> >>>> Hi, >>>> >>>> I am using MTREE and DIALPLAN modules to load lots of info to kamailio. >>>> (6 million rows). >>>> >>>> When kamailio was running with 3.2.1 (no mem_join=1 option), the used >>>> size was increasing but the process of loading the data was fast eanough. >>>> >>>> I upgraded to 3.3.2 and set mem_join=1. Now the loading process take >>>> about 10 time longer and sometimes stops kamailio from responding to >>>> traffic. >>>> >>>> Any ideas? >>>> >>>> Thanks, >>>> >>>> Uri >>>> >>> >>> >>> >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-users@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >>> >>> >>> -- >>> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda >>> >>> >> >> -- >> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda >> >> > > -- > Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda > >