Re: [SR-Users] Evapi dispatcher is consuming all the private memory

Hello,

can you try with master branch? The issue should be fixed by the commit:

* https://github.com/kamailio/kamailio/commit/d36a4b00c6becd42cfcec4fc7eb39837...

If works ok, then I will backport to stable branches.

Cheers, Daniel

On 14.08.19 12:19, José Seabra wrote:

Hello, Please find out the logs in the following link:

• https://pastebin.com/h2UNHwYC

Let me know if you need something else.

Cheers José Seabra

Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> escreveu no dia quarta, 14/08/2019 à(s) 10:59:

Hello,

that's memory status, printing details for each allocated chunk. I
could spot a lot of chunks from the parser and get header field
functions, but to be sure there is nothing else, can you set
global parameter:

mem_summary=12

and then reproduce again. The logs printed in syslog should have
smaller size, giving the summary of allocated chunks - how many
times and the size for each location in the code.

Cheers,
Daniel

On 14.08.19 11:51, José Seabra wrote:

...

Hello Daniel,
Please find out the attached zip file with the output pkg_summary.

Let me know if you need something else.

Thank you for your support

Cheers
José Seabra

Daniel-Constantin Mierla <miconda@gmail.com
<mailto:miconda@gmail.com>> escreveu no dia quarta, 14/08/2019
à(s) 10:03:

    Hello,

    can you enable pkg summary and generate it once the out of
    memory happens?

    Set memlog to lower value than debug, when the memory
    happens, do:

    https://www.kamailio.org/docs/modules/devel/modules/corex.html#corex.rpc.pkg_summary

    Then send some traffic to that process and watch the syslog.

    Alternative, enable memory summary via core parameter and
    stop kamailio when the errors happen, but you will get
    summary for all processes, so it might be harder to extract
    those log messages related the evapi process.

    Cheers,
    Daniel

    On 14.08.19 10:37, José Seabra wrote:

...

    Hello there,
    I'm using kamailio 5.2.1 with evapi module to send and
    receive events from an external application.
     
    During my stress tests to kamailio i have been noticing that
    at some point kamailio is getting out of memory:

    edge-sip-proxy[20565]ERROR: ESP_LOG: <null>: <core>
    [core/mem/q_malloc.c:291]: qm_find_free():
    qm_find_free(0x7fb8730b6010, 64); Free fragment not found!

    edge-sip-proxy[20565]: ERROR: ESP_LOG: <null>: <core>
    [core/mem/q_malloc.c:425]: qm_malloc():
    qm_malloc(0x7fb8730b6010, 64) called from core:
    core/parser/msg_parser.c: parse_headers(325), module: core;
    Free fragment not found!

    edge-sip-proxy[20565]: ERROR: ESP_LOG: <null>: <core>
    [core/parser/msg_parser.c:328]: parse_headers(): memory
    allocation error

    edge-sip-proxy[20565]: ERROR: ESP_LOG: <null>: pv
    [pv_core.c:704]: pv_get_callid(): cannot parse Call-Id header

    edge-sip-proxy[20565]: INFO: ESP_LOG: <null>:
    [evapi:connection-received] EVAPI msg received ->
    {"aor":"sip:FUZECI1-x1516@uc.fuze.domain","contact":"<sip:FUZECI1-x1516@10.225.121.206:5060;transport=UDP>"}

    edge-sip-proxy[20565]: ERROR: ESP_LOG: <null>: <core>
    [core/mem/q_malloc.c:291]: qm_find_free():
    qm_find_free(0x7fb8730b6010, 64); Free fragment not found!

    edge-sip-proxy[20565]: ERROR: ESP_LOG: <null>: <core>
    [core/mem/q_malloc.c:425]: qm_malloc():
    qm_malloc(0x7fb8730b6010, 64) called from core:
    core/parser/msg_parser.c: parse_headers(325), module: core;
    Free fragment not found!


    The process id 20565 is running out of memory as you can see
    bellow:


    output of kamcmd pkg.stats 


    entry: 14

    pid: 

    rank: -2

    used: 6058696

    free: 824

    real_used: 8387784

    total_size: 8388608

    total_frags: 30


    This process ID is related with Evapi Dispatcher



    kamcmd ps | grep 20565


    20565EvAPI Dispatcher


    After stop the sipp that was generating traffic to kamailio
    the evapi process still have all the memory consumed

    Can it be some memory leak in Evapi module?

    Best Regards
    José Seabra
    -- 
    Cumprimentos
    José Seabra

    _______________________________________________
    Kamailio (SER) - Users Mailing List
    sr-users@lists.kamailio.org <mailto:sr-users@lists.kamailio.org>
    https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

    -- 
    Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com>
    www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>



-- 
Cumprimentos
José Seabra

-- 
Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com>
www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda>

-- Cumprimentos José Seabra