@space88man pushed 1 commit.

b9b96f346eda57fb41a5a1296c8994ac8dd3237c app_python3s: initial support for free-threading Python

@space88man pushed 1 commit.

6848c5297f45639228f4793fa8bcc2bc0591d304 app_python3s: refactor GIL and thread state handling

@space88man: thanks for this work, just write a message when the PR is considered finished to review the code changes.

Hi @miconda and Kamailio devs: this PR is now ready for code review.

The main difference:

In `script_child_init` create a background task:

```python from dataclasses import dataclass import os import sys import threading import time

import KSR as KSR

# global variables corresponding to defined values (e.g., flags) in kamailio.cfg FLT_ACC = 1 FLT_ACCMISSED = 2 FLT_ACCFAILED = 3 FLT_NATS = 5

FLB_NATB = 6 FLB_NATSIPPING = 7

@dataclass class Script: rank: int pid: int

def child_task(kamailio: Script): KSR.info(f"===== child task thread {kamailio.pid}\n") ## asyncio.run(child_task_async(kamailio)) count = 0 while True: KSR.info(f"===== child task thread {kamailio.pid}:{count}\n") time.sleep(10.0) count += 1

def ksr_script_child_init(rank: int): if rank <= 0: return 1 kamailio = Script(rank, os.getpid())

flags = os.environ.get("TEST_THREADS", False) if flags: KSR.info("===== testing threads\n") child_thread = threading.Thread(target=child_task, args=[kamailio], daemon=True) child_thread.start() KSR.info("===== thread started\n") KSR.info("===== kamailio.child_init(%d:%d)\n" % (rank, kamailio.pid)) return 1

# etc etc ```

Before (master): the child task will run for 1/2 iterations at most and be blocked After (this PR): the GIL is correctly freed and the child task runs concurrently with Kamailio SIP handling

Before (only one iteration then the Python thread is blocked forever): ``` INFO: ctl [io_listener.c:214]: io_listen_loop(): using epoll_lt io watch method (config) INFO: jsonrpcs [jsonrpcs_sock.c:471]: jsonrpc_dgram_process(): a new child 0/41117 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== testing threads INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41120 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41120:0 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== thread started INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== kamailio.child_init(1:41120) INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== testing threads INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41121 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41121:0 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== thread started INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== kamailio.child_init(2:41121) ```

After (the Python background thread is running freely alongside Kamailio event loop): ``` INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== testing threads INFO: jsonrpcs [jsonrpcs_sock.c:471]: jsonrpc_dgram_process(): a new child 0/41137 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140:0 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== thread started INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== kamailio.child_init(1:41140) INFO: ctl [io_listener.c:214]: io_listen_loop(): using epoll_lt io watch method (config) INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== testing threads INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141:0 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== thread started INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== kamailio.child_init(2:41141) INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140:1 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141:1 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140:2 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141:2 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140:3 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141:3 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41140:4 INFO: <core> [core/kemi.c:107]: sr_kemi_core_info(): ===== child task thread 41141:4 ```

Some additional commits related to clang-format

@space88man pushed 1 commit.

220e12501d953d645f8265b1cae85ba37f0415ca app_python3s: make clang-format happy

the check goes commit by commit. You need to fix the format in the commits not generating a new one.

Thanks - fixed on per commit basis now

@space88man: is it required that existing kemi python scripts are updated? Are they impacted in any way? Or only those that want to executed threads in parallel can do it now?

No change to KEMI scripts - this makes threading.py work.

Closed as completed with https://github.com/kamailio/kamailio/commit/62b4ee4a0d0b62b35c8bdf67e5daf9cb...