Thanks for the prompt reply. Yes I do have TLS enabled and in fact the crash is inside one of the TCP receiver processes when it is handing a REGISTER message.

I will try with non TLS connections, however it is intermittent so the absence of crashes hasn't always been a good guide. Also I really need TLS for the client connections.

I might try something else first though. From Boto3 docs :

The boto3 module acts as a proxy to the default session

A session manages state about a particular configuration. By default a session is created for you when needed. However it is possible and recommended to maintain your own session(s) in some scenarios. 

It is also possible to manage your own session and create clients or resources from it:

# Creating your own session

session = boto3.session.Session()

sqs = session.client('sqs')

s3 = session.resource('s3')

Multithreading / Multiprocessing

--------------------------------

It is recommended to create a resource instance for each thread / process in a multithreaded or multiprocess application rather than sharing a single instance among the threads / processes. For example:

import boto3

import boto3.session

import threading

class MyTask(threading.Thread):

    def run(self):

        session = boto3.session.Session()

        s3 = session.resource('s3')

        # ... do some work with S3 ...

In the example above, each thread would have its own Boto 3 session and its own instance of the S3 resource. This is a good idea because resources contain shared data when loaded and calling actions, accessing properties, or manually loading or reloading the resource can modify this data.

So I could in __init__() initialise thus

self.session = null

self.sns_client = null

and on demand from child processes :

if not self.session:

  self.session = boto3.session.Session(region_name=MY_AWS_REGION,use_ssl=True)

if not self.client:

  self.sns_client = self.session.resource('sns')

So each child process that need it gets it's own boto3 session.

Another thing I can try is setting use_ssl=False.

Cheers

Mike

On Wed, 10 Apr 2019 at 03:34, Daniel-Constantin Mierla <miconda@gmail.com> wrote:

Hello,

do you have tls module loaded in kamailio.cfg? If yes, can you try without it, just to see if there is a conflict between our module and the boto3 client in use of libssl, because libssl creates global contexts per application, not per library/object.

Cheers,
Daniel

On 10.04.19 01:26, Michael Loughrey wrote:

Hi All,

looking for some advice regarding the proper way to initialise an AWS API boto3 client object for send SNS messages within a KEMI Python 2.7 routing script. 

I'm sure process forking has a major impact on how this works - from kamailio.cfg

     fork=yes

     children=4

-  but would greatly appreciate some guidance.

I have tried various methods to allocate an client object self.sns_client = boto3.client('sns', region_name=MY_AWS_REGION)

  1. within   __init__(), only once from module initialisation

  2. on demand within a function call by any client code e.g. called from ksr_route_request(), each creating it's own client object

 and so far all have resulted in intermittent crashes within botocore/client.py ultimately crashing thus :

Core was generated by `/usr/local/sbin/kamailio -P /usr/local/kamailio/run/kamailio.pid -f /usr/local/'.

Program terminated with signal SIGSEGV, Segmentation fault.

#0  0x00007f8f9734f754 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1

(gdb) bt

#0  0x00007f8f9734f754 in ?? () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1

#1  0x00007f8f9734f82e in X509_VERIFY_PARAM_free () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1

#2  0x00007f8f97642f5c in SSL_free () from /usr/lib/x86_64-linux-gnu/libssl.so.1.1

#3  0x00007f8f86b09c64 in PySSL_dealloc.lto_priv.5 () at ./Modules/_ssl.c:1598

#4  0x00007f8f99cbb007 in insertdict_by_entry (mp=0x7f8f841f1e88, key='_sslobj', hash=<optimized out>, ep=<optimized out>, value=<optimized out>) at ../Objects/dictobject.c:519

#5  0x00007f8f99cbe2cf in insertdict (value=None, hash=1051385741686792393, key='_sslobj', mp=0x7f8f841f1e88) at ../Objects/dictobject.c:556

#6  dict_set_item_by_hash_or_entry (value=None, ep=0x0, hash=1051385741686792393, key='_sslobj', 

    op={'server_hostname': u'sns.us-east-1.amazonaws.com', '_connected': True, '_context': <SSLContext at remote 0x7f8f84232398>, 'server_side': False, '_makefile_refs': 0, '_closed': False, '_sslobj': None, 'do_handshake_on_connect': True, 'suppress_ragged_eofs': True}) at ../Objects/dictobject.c:795

#7  PyDict_SetItem (op=<optimized out>, key=<optimized out>, value=<optimized out>) at ../Objects/dictobject.c:848

#8  0x00007f8f99becec1 in _PyObject_GenericSetAttrWithDict (obj=<optimized out>, name='_sslobj', value=None, 

    dict={'server_hostname': u'sns.us-east-1.amazonaws.com', '_connected': True, '_context': <SSLContext at remote 0x7f8f84232398>, 'server_side': False, '_makefile_refs': 0, '_closed': False, '_sslobj': None, 'do_handshake_on_connect': True, 'suppress_ragged_eofs': True}) at ../Objects/object.c:1529

#9  0x00007f8f99bed437 in PyObject_SetAttr (

    v=<SSLSocket(server_hostname=u'sns.us-east-1.amazonaws.com', _connected=True, _context=<SSLContext at remote 0x7f8f84232398>, server_side=False, _makefile_refs=0, _closed=False, _sslobj=None, do_handshake_on_connect=True, suppress_ragged_eofs=True) at remote 0x7f8f841f62a8>, name=<optimized out>, value=None) at ../Objects/object.c:1247

Here's my setup

host : Debian9/stretch

python --version

Python 2.7.13

kamailio -v

version: kamailio 5.2.2 (x86_64/linux) 67f967-dirty

flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES, MEM_JOIN_FREE

ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB

poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.

id: 67f967 -dirty

compiled on 23:44:31 Apr  8 2019 with gcc 6.3.0

Cheers

Mike

_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla -- www.asipto.com
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio World Conference - May 6-8, 2019 -- www.kamailioworld.com