Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Hello,
Does this happen immediately, or some time after you restart Kamailio? During peak traffic loads? Randomly?
-- Alex
On 9/8/20 4:53 PM, M Arqum CH wrote:
Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Regards M Arqum
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hi Alex, It happened after 3 to 4 days of restart and happen in peak hours, but the load is not too high less than 100 calls.
Thanks
On Wed, Sep 9, 2020 at 1:56 AM Alex Balashov abalashov@evaristesys.com wrote:
Hello,
Does this happen immediately, or some time after you restart Kamailio? During peak traffic loads? Randomly?
-- Alex
On 9/8/20 4:53 PM, M Arqum CH wrote:
Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Regards M Arqum
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello,
as probably mentioned already, first try to do an update to a maintained version (e.g. 5.3.6).
Try the steps descriped at “monitoring memory” in this wiki page: https://www.kamailio.org/wiki/tutorials/troubleshooting/memory
Especially look out for increases in one of the module or core section(s).
If you find out that it keeps growing in some area, after you have done an update, open an issue on our tracker.
Cheers,
Henning
-- Henning Westerholt – https://skalatan.de/blog/ Kamailio services – https://gilawa.comhttps://gilawa.com/
From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of M Arqum CH Sent: Wednesday, September 9, 2020 11:10 AM To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] Getting memory error on kamailio v5.1.2 tm:Free fragment not found!
Hi Alex, It happened after 3 to 4 days of restart and happen in peak hours, but the load is not too high less than 100 calls.
Thanks
On Wed, Sep 9, 2020 at 1:56 AM Alex Balashov <abalashov@evaristesys.commailto:abalashov@evaristesys.com> wrote: Hello,
Does this happen immediately, or some time after you restart Kamailio? During peak traffic loads? Randomly?
-- Alex
On 9/8/20 4:53 PM, M Arqum CH wrote:
Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Regards M Arqum
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.orgmailto:sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Regards M Arqum
Hi,
You can check statistics of shared memory on Kamailio. In this way, you can decide whether you need to increase your shmem or not..
Type "kamctl stats shmem" on cli of Kamailio server and share the output..
Regards Egemen U.
Get Outlook for Androidhttps://aka.ms/ghei36 ________________________________ From: sr-users sr-users-bounces@lists.kamailio.org on behalf of M Arqum CH marqumch@gmail.com Sent: Tuesday, September 8, 2020 11:53:18 PM To: sr-users@lists.kamailio.org sr-users@lists.kamailio.org Subject: [SR-Users] Getting memory error on kamailio v5.1.2 tm:Free fragment not found!
Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks -- Regards M Arqum
Hello,
the 5.1.2 is old, unmaintained at this moment, but you should always run the latest version in a series to be sure you are not affected by bugs fixed already. For example, in this case you are in 5.1.x series, so you should run 5.1.10. But it is better to upgrade to a maintained series, like 5.3 or 5.4.
Back to 5.1.x, after upgrading to 5.1.10, you can also set mem_join=1 in the config file, that can help in such situations, provided that to allocated enough shared memory for the traffic you are handling. In newer release series of kamailio mem_join is 1 by default, but not in 5.1.x.
Cheers, Daniel
On 08.09.20 23:06, egemen ulus wrote:
Hi,
You can check statistics of shared memory on Kamailio. In this way, you can decide whether you need to increase your shmem or not..
Type "kamctl stats shmem" on cli of Kamailio server and share the output..
Regards Egemen U.
Get Outlook for Android https://aka.ms/ghei36
*From:* sr-users sr-users-bounces@lists.kamailio.org on behalf of M Arqum CH marqumch@gmail.com *Sent:* Tuesday, September 8, 2020 11:53:18 PM *To:* sr-users@lists.kamailio.org sr-users@lists.kamailio.org *Subject:* [SR-Users] Getting memory error on kamailio v5.1.2 tm:Free fragment not found! Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Regards M Arqum
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Hello Denial ,
thank you for your reply , i will work on update to 5.1.10, for time being i increase the -M to 256.
Thanks
On Wed, Sep 9, 2020 at 12:43 PM Daniel-Constantin Mierla miconda@gmail.com wrote:
Hello,
the 5.1.2 is old, unmaintained at this moment, but you should always run the latest version in a series to be sure you are not affected by bugs fixed already. For example, in this case you are in 5.1.x series, so you should run 5.1.10. But it is better to upgrade to a maintained series, like 5.3 or 5.4.
Back to 5.1.x, after upgrading to 5.1.10, you can also set mem_join=1 in the config file, that can help in such situations, provided that to allocated enough shared memory for the traffic you are handling. In newer release series of kamailio mem_join is 1 by default, but not in 5.1.x.
Cheers, Daniel On 08.09.20 23:06, egemen ulus wrote:
Hi,
You can check statistics of shared memory on Kamailio. In this way, you can decide whether you need to increase your shmem or not..
Type "kamctl stats shmem" on cli of Kamailio server and share the output..
Regards Egemen U.
Get Outlook for Android https://aka.ms/ghei36
*From:* sr-users sr-users-bounces@lists.kamailio.org sr-users-bounces@lists.kamailio.org on behalf of M Arqum CH marqumch@gmail.com marqumch@gmail.com *Sent:* Tuesday, September 8, 2020 11:53:18 PM *To:* sr-users@lists.kamailio.org sr-users@lists.kamailio.org sr-users@lists.kamailio.org *Subject:* [SR-Users] Getting memory error on kamailio v5.1.2 tm:Free fragment not found!
Hello Users,
I am facing memory-related error on my Kamailio server, using this kamailio version kamailio -v version: kamailio 5.1.2 (x86_64/linux) with -m 512 -M 32 childern=16
System Memory : 24 GB System Cores: 12
On console getting the following error :
Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:286]: qm_find_free(): qm_find_free(0x7fdf51eb9000, 11320); Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: <core> [core/mem/q_malloc.c:419]: qm_malloc(): qm_malloc(0x7fdf51eb9000, 11320) called from tm: h_table.c: build_cell(320), module: tm; Free fragment not found! Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1279]: new_t(): out of mem: Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: tm [t_lookup.c:1422]: t_newtran(): new_t failed Sep 08 14:40:29 debian /usr/sbin/kamailio[99894]: ERROR: sl [sl_funcs.c:362]: sl_reply_error(): stateless error reply used: No error (2/SL)
Please guide about the possible reason of failure or its memory leakage, possible solution.
Thanks
Regards M Arqum
Kamailio (SER) - Users Mailing Listsr-users@lists.kamailio.orghttps://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda Funding: https://www.paypal.me/dcmierla
Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users