Hi everyone,

I am using Kamailio-4.1.5. Sometime kamailio was crash because memory leak.

Log when out of mem

May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: tm [t_lookup.c:1472]: t_newtran(): ERROR: t_newtran: new_t failed
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: tm [t_lookup.c:1332]: new_t(): ERROR: new_t: out of mem:
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: tm [t_lookup.c:1472]: t_newtran(): ERROR: t_newtran: new_t failed
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [ucontact.c:103]: new_ucontact(): no more shm memory
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [urecord.c:155]: mem_insert_ucontact(): failed to create new contact
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [urecord.c:547]: insert_ucontact(): failed to insert contact
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: registrar [save.c:708]: update_contacts(): failed to insert contact
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: tm [t_lookup.c:1332]: new_t(): ERROR: new_t: out of mem:
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: tm [t_lookup.c:1472]: t_newtran(): ERROR: t_newtran: new_t failed
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [ucontact.c:103]: new_ucontact(): no more shm memory
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [urecord.c:155]: mem_insert_ucontact(): failed to create new contact
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: usrloc [urecord.c:547]: insert_ucontact(): failed to insert contact
May  2 09:49:00 vnpsip01 /usr/local/sbin/kamailio[30821]: ERROR: registrar [save.c:521]: insert_contacts(): failed to insert contact

Log when kamailio was crash

May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30818]: ALERT: <core> [main.c:775]: handle_sigs(): child process 30821 exited by a signal 11
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30818]: ALERT: <core> [main.c:778]: handle_sigs(): core was not generated
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30818]: INFO: <core> [main.c:790]: handle_sigs(): INFO: terminating due to SIGCHLD
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status: heap size= 749731840
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status: used= 1110960, used+overhead=1663384, free=748068456
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status: max used (+overhead)= 1825240
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status: dumping all alloc'ed. fragments:
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      0. N  address=0x7f6af599d528 frag=0x7f6af599d4f8 size=1024 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      1. N  address=0x7f6af599d988 frag=0x7f6af599d958 size=256 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30828]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      2. N  address=0x7f6af599dae8 frag=0x7f6af599dab8 size=176 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: counters.c: cnt_hash_add(339)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      3. N  address=0x7f6af599dbf8 frag=0x7f6af599dbc8 size=240 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: counters.c: cnt_hash_add(339)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      4. N  address=0x7f6af599dd48 frag=0x7f6af599dd18 size=128 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: counters.c: init_counters(129)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      5. N  address=0x7f6af599de28 frag=0x7f6af599ddf8 size=16 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: modparam.c: set_mod_param_regex(124)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      6. N  address=0x7f6af599de98 frag=0x7f6af599de68 size=128 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      7. N  address=0x7f6af599df78 frag=0x7f6af599df48 size=48 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: route_add(159)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      8. N  address=0x7f6af599e008 frag=0x7f6af599dfd8 size=16 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: init_rlist(181)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:      9. N  address=0x7f6af599e078 frag=0x7f6af599e048 size=128 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     10. N  address=0x7f6af599e158 frag=0x7f6af599e128 size=48 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: route_add(159)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     11. N  address=0x7f6af599e1e8 frag=0x7f6af599e1b8 size=16 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: init_rlist(181)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     12. N  address=0x7f6af599e258 frag=0x7f6af599e228 size=128 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     13. N  address=0x7f6af599e338 frag=0x7f6af599e308 size=48 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: route_add(159)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     14. N  address=0x7f6af599e3c8 frag=0x7f6af599e398 size=16 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: route.c: init_rlist(181)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     15. N  address=0x7f6af599e438 frag=0x7f6af599e408 size=128 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status: heap size= 749731840
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status: used= 1110768, used+overhead=1663192, free=748068648
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status: max used (+overhead)= 1825240
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status: dumping all alloc'ed. fragments:
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status:      0. N  address=0x7f6af599d528 frag=0x7f6af599d4f8 size=1024 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status:      1. N  address=0x7f6af599d988 frag=0x7f6af599d958 size=256 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30833]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30827]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30828]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status: heap size= 749731840
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status: used= 1110768, used+overhead=1663192, free=748068648
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status: max used (+overhead)= 1825240
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status: dumping all alloc'ed. fragments:
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30828]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:      0. N  address=0x7f6af599d528 frag=0x7f6af599d4f8 size=1024 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status: heap size= 749731840
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status: used= 1110768, used+overhead=1663192, free=748068648
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status: max used (+overhead)= 1825240
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status: dumping all alloc'ed. fragments:
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status:      0. N  address=0x7f6af599d528 frag=0x7f6af599d4f8 size=1024 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:      1. N  address=0x7f6af599d988 frag=0x7f6af599d958 size=256 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:           alloc'd from <core>: str_hash.h: str_hash_alloc(69)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30834]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:      2. N  address=0x7f6af599dae8 frag=0x7f6af599dab8 size=176 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30830]: NOTICE: qm_status:           alloc'd from <core>: counters.c: cnt_hash_add(339)
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status:          start check=f0f0f0f0, end check= c0c0c0c0, abcdefed
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30832]: NOTICE: qm_status:      1. N  address=0x7f6af599d988 frag=0x7f6af599d958 size=256 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30834]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30823]: NOTICE: qm_status:     16. N  address=0x7f6af599e518 frag=0x7f6af599e4e8 size=48 used=1
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30834]: NOTICE: qm_status: (0x7f6af5968010):
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30822]: INFO: <core> [main.c:841]: sig_usr(): INFO: signal 15 received
May  2 10:50:41 vnpsip01 /usr/local/sbin/kamailio[30822]: NOTICE: <core> [main.c:852]: sig_usr(): Memory status (pkg):

Could you help me to fix this problem!

Thanks so much!

--

---------------------------

Bui Thi Minh Yen