sr-dev August 2021

sr-dev@lists.kamailio.org

19 participants
267 discussions

[kamailio/kamailio] Kamailio was not accepting TLS connections (#2796)

by aatif218

we are getting the following errors in Kamailio logs **kamailio.log** Jun 29 08:08:14 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: ERROR: <core> [core/tcp_main.c:4451]: handle_tcpconn_ev(): io_watch_del(3) failed: for 0x7f524f7862d0, fd 30406 Jun 29 08:08:14 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: ERROR: <core> [core/tcp_main.c:4451]: handle_tcpconn_ev(): io_watch_del(3) failed: for 0x7f525a6c5f90, fd 30407 Jun 29 08:08:14 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: ERROR: <core> [core/tcp_main.c:4451]: handle_tcpconn_ev(): io_watch_del(3) failed: for 0x7f52449e0978, fd 30415 Jun 29 08:06:27 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: CRITICAL: <core> [core/io_wait.h:596]: io_watch_del(): invalid fd 30355, not in [0, 4607) Jun 29 08:06:27 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: CRITICAL: <core> [core/io_wait.h:596]: io_watch_del(): invalid fd 30356, not in [0, 4607) Jun 29 08:06:27 fep-1 /usr/local/fep-kamailio/sbin/kamailio[14111]: CRITICAL: <core> [core/io_wait.h:596]: io_watch_del(): invalid fd 30358, not in [0, 4607) **Description** On checking this issue we found that Kamailio was not accepting TLS connections. We were getting an error which is mentioned above. We double-checked TLS and TCP connections and they were normal. We have 5000+ TLS connections and the max connection limit is 30,000. So we were not exceeding the TLS connection limit. We also verified open_files_limits. It is around 1 million which is more than enough. **Kamailio Version** - output of kamailio -v version: kamailio 5.4.2 (x86_64/linux) f8885c flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. id: f8885c **Operating System:** NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2796

3 years, 2 months

[kamailio/kamailio] kamailio log socket error (#2806)

by nihaomatmat

I have some problems。the kamailio version is 5.3.0. My linux version is centos 7. the kamailio log: ERROR: <core> [core/tcp_main.c:3508]: send_fd_queue_run(): send_fd failed on socket 63 , queue entry 1, retries 90, connection 0x7ff46e6b1b60, tcp socket 218, errno=11 (Resource temporarily unavailable) ERROR: {1 12529 MESSAGE 359bf0410a7e471c950623e3f7a3b17b} <core> [core/kemi.c:94]: sr_kemi_core_err(): 202->408 CRITICAL: <core> [core/tcp_main.c:4218]: send2child(): tcp child 6, socket 65: queue full, 291 requests queued (total handled 10665748) This problem does not happen every time。When the number of user registrations increases significantly, errors will appear。I tried to set the parameter tcp_children is adjusted to 8, it will not appear frequently, but when the number of user registrations increases to about 5000, errors will still appear. My kamailio.cfg global parameters are set as follows：tls_max_connections=65536,tcp_max_connections=65536,open_max_files=65536。 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2806

3 years, 2 months

[kamailio/kamailio] ims_charging: fix charging reply route with kemi (#2836)

by Riccardo-78

#### Pre-Submission Checklist    - [x] Commit message has the format required by CONTRIBUTING guide - [x] Commits are split per component (core, individual modules, libs, utils, ...) - [x] Each component has a single commit (if not, squash them into one commit) - [x] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [x] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [x] PR should be backported to stable branches - [x] Tested changes locally - [ ] Related to issue #XXXX (replace XXXX with an open issue number) #### Description  This patch fixes route callback for ccr responses when using kemi as configuration engine. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/2836 -- Commit Summary -- * ims_charging: fix charging reply route with kemi -- File Changes -- M src/modules/ims_charging/ims_charging_mod.c (30) M src/modules/ims_charging/ims_ro.c (35) M src/modules/ims_charging/ims_ro.h (2) -- Patch Links -- https://github.com/kamailio/kamailio/pull/2836.patch https://github.com/kamailio/kamailio/pull/2836.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2836

3 years, 2 months

Bullseye builds

by George Diamantopoulos

Hello all, Debian 11 "Bullseye" is to be released this Saturday. Do you expect to have binaries available for it on deb.kamailio.org anytime soon following the release? Thanks! BR, George

3 years, 2 months

[kamailio/kamailio] OpenSSL 1.1.1 and HSM keys incoming patches (#2839)

by aalba6675

### Description When using HSM keys (via OpenSSL engine) the engine and private keys are loaded in the child processes since PKCS#11 modules rarely survive `fork()`. With OpenSSL 1.1.1 and the call to `OPENSSL_init_ssl()` in `tls_init.c` the engine linked-list is now initialized in the master process. Subsequently the technique used in `tls_init.c:660 CONF_modules_load_file()` will fail as each child process manipulates the linked-list. Even if the engine linked-list manipulation itself is protected by global locks; traversing the linked-list will encounter invalid memory locations as the nodes are insert by other child processes. This issue is filed as a marker; I will be providing patches to change the way HSM private keys are loaded in the child process to avoid linked-list corruption. ### Troubleshooting - use soft (PEM) keys — no issue as OpenSSL engine is not relevant in this context #### Reproduction - configure TLS with OpenSSL engine key(e.g pkcs11 engine with SoftHSM keys) ### Additional Information Root cause: the function `CONF_modules_load_file()` will configure OpenSSL engines from a OpenSSL configuration file but it also adds each engine to the global linked list. Unfortunately there is no OpenSSL function to reset the linked-list in the child process. It seems to be initialized once at start-up: the linked-list head/tail pointers are static variables and their initialization is protected by OpenSSL's `RUN_ONCE` mechanism, so it doesn't seem it supported mechanism to have them reinitialized after `fork()`. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/2839

3 years, 2 months

[kamailio/kamailio] tls: fix OpenSSL engine in child processes (#2840)

by aalba6675

tls_init.c calls OPENSSL_init_ssl(); this initializes the global engine linked-list and this cannot be reset in the child. To avoid linked-list corruption we manually instantiate the engine object required for loading private keys instead of relying on CONF_modules_load_file(). Updates to doc/. Addresses #2839   #### Pre-Submission Checklist    - [X] Commit message has the format required by CONTRIBUTING guide - [X] Commits are split per component (core, individual modules, libs, utils, ...) - [X] Each component has a single commit (if not, squash them into one commit) - [X] No commits to README files for modules (changes must be done to docbook files in `doc/` subfolder, the README file is autogenerated) #### Type Of Change - [X] Small bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds new functionality) - [ ] Breaking change (fix or feature that would change existing functionality) #### Checklist:  - [X] PR should be backported to stable branches - [X] Tested changes locally - [X] Related to issue #2839 #### Description The call to `OPENSSL_init_ssl()` in `tls_init.c` results in the creation of the engine linked-list in the parent. This affects per-child engine private keys as there is no api to reinitialize the engine linked-list in the child. This PR removes the call to `CONF_modules_load_file()` which causes linked-list corruption and replaces the initialization of engine private keys in the child with with other api calls which do not manipulate global objects. You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/2840 -- Commit Summary -- * tls: fix OpenSSL engine in child processes -- File Changes -- M src/modules/tls/doc/hsm_howto.xml (15) M src/modules/tls/tls_mod.c (93) -- Patch Links -- https://github.com/kamailio/kamailio/pull/2840.patch https://github.com/kamailio/kamailio/pull/2840.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/2840

3 years, 2 months