22 Mar
2010
22 Mar
'10
7:14 p.m.
I have some questions regarding the shared memory. I'm working with the version 1.5.2
and
its environment is quite simple. It does'nt storage any informations in the memory,
like dialplan,
contacts of subscribers, carrierroute, LCR and etc. The only purpose of this server is
being a
redirect server. Some calls have several possible destinations, reaching easily 12
contacts.
The config.h's constant MAX_BRANCHES is set in 12, the default number. Ok.
Like I said, almost the requests have more than 12 possible contacts. When it happens,
Kamailio
logs these messages:
/usr/local/kamailio-1.5.2/sbin/kamailio[26188]: ERROR:core:append_branch: max nr of
branches exceeded
/usr/local/kamailio-1.5.2/sbin/kamailio[26188]: ERROR:pv:pv_set_branch: append_branch
action failed
Ok. It is not a problem. It is an error but it does not stop the system. The SIP messages
still flowing
normally. The server works for several days, 24/7 without functional errors, receiving
five hundred thousand requests per day . But, usually at the weekends and the clock tells
2:30 am,
the company's cell phone rings bringing a message: 'the system is down'.
Kamailio shows:
/usr/local/kamailio-1.5.2/sbin/kamailio[25492]: ERROR:core:add_avp: no more shm mem
/usr/local/kamailio-1.5.2/sbin/kamailio[25492]: ERROR:exec:exec_avp: unable to add avp
Kamailio stills running but it is not possible to route any request. All the new incoming
calls generates the same error: 'no more shm mem'. I changed the size of shared
memory,
running Kamailio with '-m 64', but the server stopped again after some days. I see
the options:
- increase MAX_BRANCHES
- increase again 'm' parameter (but it will just add more some days in the running
time)
- reduce the maximum number of contacts to 11 of the script that returns them to Kamailio
So, my doubts are:
- is it possible to have a memory leak when the message "max nr of branches
exceeded" is showed?
- changing '-m parameter' to a higher value, like 256, could affect the
performance?
tks friends
bruno machado
____________________________________________________________________________________
Veja quais são os assuntos do momento no Yahoo! +Buscados
http://br.maisbuscados.yahoo.com
22 Mar
22 Mar
8:09 p.m.
On Monday 22 March 2010, Bruno Machado wrote:
I have some questions regarding the shared memory.
I'm working with the
version 1.5.2 and its environment is quite simple. It does'nt storage any
informations in the memory, like dialplan, contacts of subscribers,
carrierroute, LCR and etc. The only purpose of this server is being a
redirect server. Some calls have several possible destinations, reaching
easily 12 contacts. The config.h's constant MAX_BRANCHES is set in 12, the
default number. Ok. Like I said, almost the requests have more than 12
possible contacts. When it happens, Kamailio logs these messages:
/usr/local/kamailio-1.5.2/sbin/kamailio[26188]: ERROR:core:append_branch:
max nr of branches exceeded
/usr/local/kamailio-1.5.2/sbin/kamailio[26188]: ERROR:pv:pv_set_branch:
append_branch action failed
[..]
/usr/local/kamailio-1.5.2/sbin/kamailio[25492]: ERROR:core:add_avp: no more
shm mem /usr/local/kamailio-1.5.2/sbin/kamailio[25492]:
ERROR:exec:exec_avp: unable to add avp
Kamailio stills running but it is not possible to route any request. All
the new incoming calls generates the same error: 'no more shm mem'. I
changed the size of shared memory, running Kamailio with '-m 64', but the
server stopped again after some days. I see the options:
- increase MAX_BRANCHES
- increase again 'm' parameter (but it will just add more some days in the
running time) - reduce the maximum number of contacts to 11 of the script
that returns them to Kamailio
So, my doubts are:
- is it possible to have a memory leak when the message "max nr of branches
exceeded" is showed? - changing '-m parameter' to a higher value, like
256, could affect the performance?
Hi Bruno,
this looks indeed like an memory leak in the code path that is executed during
the error. Also possible is that there is (really small) mem leak in another
code path, that gets executed a lot.
Another (unlikely) option could be that this problem is caused from shared
memory fragmentation, as you noticed that the server gets some traffic. How
often this error happens? Do you can clearly correlate this to the error logs
you notice?
Anyway, you should increase the shared memory setting to 512M or higher,
depending on the system memory. It has no negative performance impact.
Henning
23 Mar
23 Mar
5:02 p.m.
Hi Henning
Thanks for your answer. This memory error happened in a interval of almost 10 days.
Considering the high traffic that the server handles, we can say that it is a
lot of time. Following your hint, I set the shared memory to work with 512 megabytes.
If we consider that the usage of memory is a linear function of the number of calls,
the server will work ininterruptly for more than 70 days.
I have been looking for a way to monitor how much of shared memory was already used.
But I didn't find anything, except the troubleshooting section on the Devel Docs.
I'm not using Kamailio v3 yet, but I saw that it has a memory manager. I think that
these features (Doug Lea and LL) are going to be used mainly for developers.
The third one, 'print memory status summary in debug mode', is more oriented to
users like me. It is what I think.
Despite anything, it is already time to migrate to version 3.
tks
bruno machado
____________________________________________________________________________________
Veja quais são os assuntos do momento no Yahoo! +Buscados
http://br.maisbuscados.yahoo.com
5:21 p.m.
On Tuesday 23 March 2010, Bruno Machado wrote:
Thanks for your answer. This memory error happened in
a interval of almost
10 days. Considering the high traffic that the server handles, we can say
that it is a lot of time.
Hello Bruno,
10 days is not that long, this looks more like a memory leak.
Following your hint, I set the shared memory to
work with 512 megabytes. If we consider that the usage of memory is a
linear function of the number of calls, the server will work ininterruptly
for more than 70 days.
I have been looking for a way to monitor how much of shared memory was
already used. But I didn't find anything, except the troubleshooting
section on the Devel Docs. I'm not using Kamailio v3 yet, but I saw that
it has a memory manager. I think that these features (Doug Lea and LL) are
going to be used mainly for developers. The third one, 'print memory
status summary in debug mode', is more oriented to users like me. It is
what I think.
Kamailio has (more or less) the same internal memory manager. You could use
the statistics API (kamctl fifo get_statistics all). This should give you
plenty of statistics, including the shared memory usage.
Regards,
Henning
5365
days inactive
5366
days old
4 comments
2 participants
participants (2)
-
Bruno Machado -
Henning Westerholt