Re: [SR-Users] TLS not enough memory issue with git master

Looks like a lot of connections being open, can you get the output for: kamcmd core.tcp_info kamcmd tls.info Cheers, Daniel On 17/11/15 14:59, Anthony Messina wrote: > Attached. -A > > On Tuesday, November 17, 2015 02:50:21 PM Daniel-Constantin Mierla wrote: >> Can you run the following commands: >> >> kamcmd cfg.set_now_int core memlog 1 >> kamcmd corex.shm_summary >> >> Then grab the log messages from syslog related to shared memory summary >> and send them over here. >> >> Cheers, >> Daniel >> >> On 17/11/15 14:31, Anthony Messina wrote: >>> After I reported last night, I restarted Kamailio and even though the 5 >>> UACs did nothing but ensure they had a registration overnight, this >>> morning the issue has recurred. The following is the output you >>> requested. Not sure how the memory is being used up by Kamailio. >>> >>> # kamctl stats shmem >>> shmem:fragments = 181 >>> shmem:free_size = 8922584 >>> shmem:max_used_size = 58243792 >>> shmem:real_used_size = 58186280 >>> shmem:total_size = 67108864 >>> shmem:used_size = 54346088 >>> >>> On Tuesday, November 17, 2015 09:03:24 AM Daniel-Constantin Mierla

>>>> As you are using the master branch (development), do you run latest >>>> version? >>>> >>>> Can you look at available shared memory? >>>> >>>> kamctl stats shmem >>>> >>>> Check it over time and see if the free memory is decreasing. >>>> >>>> Cheers, >>>> Daniel >>>> >>>> On 17/11/15 00:44, Anthony Messina wrote: >>>>> I have noticed the following issue which began with builds somewhere >>>>> between git master commits bff0a08 and 6173ef7. I did not see this >>>>> issue >>>>> with my previous builds and haven't been able to pin down the problem, >>>>> which is why I haven't formally filed a bug. >>>>> >>>>> Any help or guidance is appreciated, because this has crippled my use >>>>> of >>>>> Kamailio. Only a restart enables it to work again until the issue >>>>> recurs. >>>>> >>>>> ERROR: tls [tls_server.c:189]: tls_complete_init(): tls: ssl bug #1491 >>>>> workaround: not enough memory for safe operation: 8870536 >>>>> ERROR: <core> [tcp_read.c:1303]: tcp_read_req(): ERROR: tcp_read_req: >>>>> error >>>>> reading >>>>> >>>>> I currently build against and run openssl-1.0.1k-12.fc22.x86_64. >>>>> >>>>> I have a very small operation and the only change on the operational >>>>> side >>>>> is that all 5 of my mobile UACs (yes, that's all) have switched from >>>>> CSipSimple/Android to Zoiper/Android, which doesn't yet have support >>>>> for >>>>> client-side certificates so verify_certificate and require_certificate >>>>> are >>>>> off for both the server and client config. >>>>> >>>>> The server is started with: >>>>> /usr/sbin/kamailio -P /run/kamailio/kamailio.pid -m 64 -M 8 >>>>> >>>>> I have tried modifying the shared mem to 128 but the issue still >>>>> occurs. >>>>> >>>>> Even right now, I am seeing the error when only one UAC has >>>>> established >>>>> a >>>>> TLS connection: >>>>> >>>>> # kamcmd tls.list >>>>> { >>>>> >>>>> id: 572 >>>>> timeout: 3475 >>>>> src_ip: 10.77.79.156 >>>>> src_port: 58688 >>>>> dst_ip: 10.77.79.3 >>>>> dst_port: 5061 >>>>> cipher: ECDHE-RSA-RC4-SHA SSLv3 Kx=ECDH Au=RSA Enc=RC4(128) >>>>> Mac=SHA1 >>>>> ct_wq_size: 0 >>>>> enc_rd_buf: 0 >>>>> flags: 2 >>>>> state: established >>>>> >>>>> } >>>>> >>>>> # kamailio.cfg >>>>> enable_tls=yes >>>>> loadmodule "tls.so" >>>>> modparam("tls", "connection_timeout", 60) >>>>> #modparam("tls", "tls_log", 1) >>>>> #modparam("tls", "tls_debug", 1) >>>>> #modparam("tls", "low_mem_threshold1", -1) >>>>> #modparam("tls", "low_mem_threshold2", 0) >>>>> modparam("tls", "session_cache", 1) >>>>> >>>>> # tls.cfg >>>>> [server:default] >>>>> method = TLSv1+ >>>>> verify_certificate = no >>>>> require_certificate = no >>>>> private_key = /etc/kamailio/example.org.key.pem >>>>> certificate = /etc/kamailio/example.org.crt.pem >>>>> server_name = example.org >>>>> cipher_list = >>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA- >>>>> AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA- >>>>> AE >>>>> S >>>>> 256- >>>>> SHA:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-GCM >>>>> - >>>>> SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:RC4 >>>>> - >>>>> SHA:AES256-GCM-SHA384:AES256-SHA256:CAMELLIA256-SHA:ECDHE-RSA-AES128- >>>>> SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:CAMELLIA128- >>>>> SHA:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK >>>>> >>>>> [client:default] >>>>> method = TLSv1+ >>>>> verify_certificate = no >>>>> require_certificate = no >>>>> private_key = /etc/kamailio/example.org.key.pem >>>>> certificate = /etc/kamailio/example.org.crt.pem >>>>> cipher_list = >>>>> ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA- >>>>> AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-RC4-SHA:ECDHE-RSA- >>>>> AE >>>>> S >>>>> 256- >>>>> SHA:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-GCM >>>>> - >>>>> SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:RC4 >>>>> - >>>>> SHA:AES256-GCM-SHA384:AES256-SHA256:CAMELLIA256-SHA:ECDHE-RSA-AES128- >>>>> SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:CAMELLIA128- >>>>> SHA:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK >>>>> >>>>> >>>>> Thanks. -Anthony