Still now showing the debug symbols for libssl/libcrypto:

  1. 0x00007fdc7d07754d in __lll_lock_wait () from /lib64/libpthread.so.0
  2. #0 0x00007fdc7d07754d in __lll_lock_wait () from /lib64/libpthread.so.0
  3. No symbol table info available.
  4. #1 0x00007fdc7d0743d2 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0
  5. No symbol table info available.
  6. #2 0x00007fdc737f2c59 in CRYPTO_THREAD_write_lock () from /usr/local/lib64/kamailio/modules/tlsa.so
  7. No symbol table info available.
  8. #3 0x00007fdc7379e32e in CRYPTO_free_ex_data () from /usr/local/lib64/kamailio/modules/tlsa.so
  9. No symbol table info available.
  10. #4 0x00007fdc73708ae2 in SSL_free () from /usr/local/lib64/kamailio/modules/tlsa.so
  11. No symbol table info available.

How are these libs installed?

Cheers,
Daniel

On 29.11.22 10:38, Ihor Olkhovskyi wrote:

Daniel,

Thanks for the tip, recompiled and got a new trace

https://pastebin.com/NxCXA1gT

Not sure what happened, but sometimes running kamctl trap restores Kamailio functionality to accept connections.

Thanks in advance!

Le 29/11/2022 à 08:07, Daniel-Constantin Mierla a écrit :

Hello,

is libssl/libcrypto compiled/installed with debugging symbols? They are not shown in the backtrace.

Cheers,
Daniel

On 28.11.22 17:38, Igor Olhovskiy wrote:
Hello,

I'm continuing investigations on Kamailio and stress test. Got it again in the state where it's not accepting any new TCP/TLS connections (UDP still works though), but all looks good from lsof/netnstat part, like system is not reporting any zombie connections. Restart of Kamailio process helps

This time I got output of kamctl trap

Put it here: https://pastebin.com/iYrNZ8U9

kamailio --version
version: kamailio 5.6.2 (x86_64/linux) 54a9c1
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 54a9c1
compiled on 14:01:01 Oct 18 2022 with gcc 4.8.5

It's statically linked with tlsa pointing on openssl-1.1.1q

Settings related to TLS are:

fork=yes
children=4
tcp_children=12
enable_tls=yes

tcp_accept_no_cl=yes
tcp_max_connections=63536
tls_max_connections=63536
tcp_accept_aliases=no
tcp_async=yes
tcp_connect_timeout=10
tcp_conn_wq_max=63536
tcp_crlf_ping=yes
tcp_delayed_ack=yes
tcp_fd_cache=yes
tcp_keepalive=yes
tcp_keepcnt=3
tcp_keepidle=30
tcp_keepintvl=10
tcp_linger2=30
tcp_rd_buf_size=80000
tcp_send_timeout=10
tcp_wq_blk_size=2100
tcp_wq_max=10485760
open_files_limit=63536


Can you please help to read gdb output and understand where I missed in config?

Thanks in advance!


--
Best regards,
Ihor (Igor)

__________________________________________________________
Kamailio - Users Mailing List - Non Commercial Discussions
sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender!
Edit mailing list options or unsubscribe:
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
-- 
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda