Hi,
== Short story:
Currently, there is a (corner case) performance issue in kamailio when lots of fragments of about the same size are freed in a short time.
== Long story:
I've recently hit a wall using kamailio, dealing with ~150k TLS connections in parallel: when I stop my injector (sipp), all the connections are closed at the same time. The TCP main process took ~20 minutes to dispose of all the objects, and was unable to accept any new connection in the meantime.
The bottleneck lies in the shm_free() code (whether f_malloc or q_malloc is used), when a free fragment is be inserted in an ordered linked list (which is O(n) if n is the size of the list). Enabling mem_join did help a little (less fragments, shorter lists), and the 150k disconnections scenario went down to ~5 minutes unavailability for the TCP main process. But this is still big.
== TLSF:
I looked for alternatives to f/q_malloc, and I found TLSF:
This allocator has O(1) malloc and free even in every case. This means no surprise like the one I had previously.
I took the implementation found here:
and integrated it in kamailio. My 150k disconnections scenario went down to 12 seconds, and the fragmentation is as good as q_malloc() with mem_join.
I've published a branch with TLSF here:
* https://github.com/kamailio/kamailio/commits/tmp/TLSF_malloc
What do you think of integrating it in kamailio?
By the way, I've tried to play with dl_malloc, ll_malloc and sf_malloc, but none of these currently compile. Should we maintain them or remove them from the codebase?