Hello all,
I am currently looking into issue
https://github.com/kamailio/kamailio/issues/3823
regarding the tls.reload and the constant increase in memory usage when it's called.
I tried to look something up related to CRL and related functions for what causing it but
nothing to obvious. The only culprit i could find was this function call in
SSL_verify_load_locations in
load_crl()<https://github.com/kamailio/kamailio/blob/9c2705b566103f4fad0…79>.
But the same exact function is also called in load_ca_list() with no such behaviour.
Next thing i notice, is that on the first rpc call of tls.reload, there is a
"significant" memory allocation.
What i mean by this is that when a fresh kamailio (5.8) is started up (with -m 7, for low
shared memory availability to try and replicate the issue and children=2) and then check
free shared memory usage using `kamcmd core.shmmem` is around 2196096 (bytes i am
assuming) for my system.
On the first call of tls.reload, it drops to 539248. That's a heavy drop so i started
digging why on so much allocation is caused. When loading a tls configuration,
ksr_fix_domain<https://github.com/kamailio/kamailio/blob/9c2705b566103f4…
allocates a new SSL_CTX per process. From what i am seeing this SSL_CTX, is basically the
same for each one, with same configuration without any difference on settings.
So, what i tried is to allocated it once (see tls_reload branch in my
repo<https://github.com/kamailio/kamailio/compare/master...xkaraman:kama…)d>),
and just share this context to all of the process and the memory used is way less and also
the unbounding memory seems to fixed or at least it's slowed that's barely
visible.
What are your thoughts on these? Is this something we can done? Is the assumption that tls
config will and be the same for every process or can one somehow differentiate from
another?
Thanks,
Xenofon
Some numbers for anyone interested:
Before the patch 5.8 branch and with settings such as -m 7 (shared memory) and children=2
(with children=8 it doesn't even start, failed to load cyphers in SSL_CTX):
Fresh kamailio start shared memory: total: 7340032
free: 2196096
After one tls.reload: free: 539248
After 100 tls.reload: free: 483776
After patch on master (kamailio -m 7 and children=2) (with children=8 it starts
normally):
Fresh kamailio start shared memory: total: 7340032
free: 3772000
After one tls.reload: free: 3690192
After 100 tls.reload: free: 3681056
[
https://repository-images.githubusercontent.com/15101579/2d895000-e695-11e9…
Comparing kamailio:master...xkaraman:tls_reload ·
kamailio/kamailio<https://github.com/kamailio/kamailio/compare/master...…
Kamailio - The Open Source SIP Server for large VoIP and real-time communication platforms
- - Comparing kamailio:master...xkaraman:tls_reload · kamailio/kamailio
github.com
[
https://opengraph.githubassets.com/a209e4474f30e5fb4ce474f928991f0fba268f02…
kamailio/src/modules/tls/tls_domain.c at 9c2705b566103f4fad0cad63bc1e36ba3cca621f ·
xkaraman/kamailio<https://github.com/xkaraman/kamailio/blob/9c2705b56610…
Kamailio - The Open Source SIP Server for large VoIP and real-time communication platforms
- - xkaraman/kamailio
github.com
[
https://repository-images.githubusercontent.com/15101579/2d895000-e695-11e9…
kamailio/src/modules/tls/tls_domain.c at 9c2705b566103f4fad0cad63bc1e36ba3cca621f ·
kamailio/kamailio<https://github.com/kamailio/kamailio/blob/9c2705b56610…
Kamailio - The Open Source SIP Server for large VoIP and real-time communication platforms
- - kamailio/kamailio
github.com
[
https://opengraph.githubassets.com/f4de5f69f56ea5c12c56343428618441242c2518…
Memory usage increases everytime tls.reload is executed · Issue #3823 ·
kamailio/kamailio<https://github.com/kamailio/kamailio/issues/3823>
Description We are using Kamailio 5.7.4 on Debian 12 (from
http://deb.kamailio.org/kamailio57) with rtpengine as an Edgeproxy for our clients. The
instance terminates SIP/TLS (with Cliencertificate...
github.com