Hello,
On 05/19/2009 04:25 PM, Christian Koch wrote:
Hello Daniel,
Daniel-Constantin Mierla schrieb:
we're using kamailio 1.5.1 with modules htable and perl to generate CDRs. Using htable we save some variables which are dumped to file using the perl module at the end of the call. We did a small stress test with 5 calls in parallel (each call about 20 seconds) and we got after about 6 hours the following error:
ERROR:core:pv_sprintf: pv_sprintf: Memory exhausted!this is system memory error. Has nothing to do with pkg memory used by Kamailio. Seems that perl module allocates some memory and forgets to free it. Do you see other errors in your syslog?
One memleak I could spot is that the allocated buffer to print the output in pv_sprintf is not freed if the formatted string is bad. Then I see that the output buffer is duplicated with strdup and it is freed. So seems there are some system memory leaks in perl module, indeed.
Are you sure this is system memory?
you are right, somehow I was blind when I checked the pv_sprintf, it is clear pkg_malloc().
We changed PKG_MEM_POOL_SIZE from 4 to 8MB and a stress test didn't show any problems for more then 12 hours. Then we changed it to 0.5MB and the problem occured after about 3 hours again wiht the same error message. The error message in pv_sprintf() occurs when pkg_malloc() returns NULL, so this should be a PKG issue?!? Currently we're running a test with 0.5MB of pkg memory, but we've fixed the error in pv_sprintf where the format is wrong and the buffer is not freed. In the test this morning (with 0.5MB) we had the message "wrong format" only once, and in all the other tests the message didn't show up. So we don't think this fixes the problem. Do you think only one missing pkg_free() (like in the last test) can corrupt the memory so this error occurs?
Now that the issue is pkg, you can compile and run in mem debug mode so we can detect easier where the leak is, check this: http://www.kamailio.org/dokuwiki/doku.php/troubleshooting:memory
Cheers, Daniel