### Description
Attempting to extract SIP message payload using Python3 KEMI fails with exception if the payload contains 8-bit ISO 8859-1 encoded characters.
According to Kamailio docs, KEMI functions return either string (which for Python is UTF-8), integer or null, which means 8-bit/binary payloads cannot be handled at all? This seems to go against the [SIP spec](https://www.rfc-editor.org/rfc/rfc5621.html#section-3.2), which states that SIP is 8-bit safe:
> SIP messages can carry binary message bodies such as legacy
> signalling objects [[RFC3204](https://www.rfc-editor.org/rfc/rfc3204)]. SIP proxy servers are 8-bit safe.
> That is, they are able to handle binary bodies. Therefore, there is
> no need to use encodings such as base64 to transport binary bodies in
> SIP messages. Consequently, UAs SHOULD use the binary transfer
> encoding [[RFC4289](https://www.rfc-editor.org/rfc/rfc4289)] for all payloads in SIP, including binary
> payloads. The only case where a UA MAY use a different encoding is
> when transferring application data between applications that only
> handle a different encoding (e.g., base64).
### Troubleshooting
#### Reproduction
Attempt to extract SIP message payload e.g. `KSR.pv.get("$msg(body)")` when the body has non-ASCII or UTF-8 characters, such as 'é'.
#### Debugging Data
#### Log Messages
(Proprietary specifics redacted)
```
Sep 14 22:40:09 localhost kamailio[772]: 2(65) CRITICAL: \{1 2 MESSAGE Oy9m3MMLFwBKK_EQC-WS1g..} <core> [core/kemi.c:136]: sr_kemi_core_crit(): Exception during routing: Traceback (most recent call last):
Sep 14 22:40:09 localhost kamailio[772]: File "xxx.py", line xxx, in xxx
Sep 14 22:40:09 localhost kamailio[772]: body = KSR.pv.get("$msg(body)")
Sep 14 22:40:09 localhost kamailio[772]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 0: invalid continuation byte
```
#### SIP Traffic
Full SIP message contains proprietary information. SIP message body for demonstration in above log is a single character 'é'). An otherwise identical message with body 'e' will instead decode successfully.
### Possible Solutions
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.6.2 (x86_64/linux)
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 9.4.0
```
* **Operating System**:
```
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Linux 31e2ff2f5338 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3574
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/3574(a)github.com>
From https://kamailio.sipwise.com/job/kamailiodev-nightly-binaries/architecture=…
```
cdp_tls.c: In function 'init_ssl_methods':
cdp_tls.c:52:54: error: 'TLS1_3_VERSION' undeclared (first use in this function)
tls_methods[TLS_USE_TLSv1_3_cli - 1].TLSMethodMin = TLS1_3_VERSION;
^~~~~~~~~~~~~~
cdp_tls.c:52:54: note: each undeclared identifier is reported only once for each function it appears in
../../Makefile.rules:100: recipe for target 'cdp_tls.o' failed
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/commit/defae2551f8839c5397523f135293fb…
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/commit/defae2551f8839c5397523f135293fb2a4056704/127741443(a)github.com>
From https://kamailio.sipwise.com/job/kamailiodev-nightly-binaries/architecture=…
```
cdp_tls.c: In function 'init_ssl_methods':
cdp_tls.c:21:49: warning: implicit declaration of function 'TLS_client_method' [-Wimplicit-function-declaration]
tls_methods[TLS_USE_TLSv1_cli - 1].TLSMethod = TLS_client_method();
^
cdp_tls.c:21:47: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_cli - 1].TLSMethod = TLS_client_method();
^
cdp_tls.c:24:49: warning: implicit declaration of function 'TLS_server_method' [-Wimplicit-function-declaration]
tls_methods[TLS_USE_TLSv1_srv - 1].TLSMethod = TLS_server_method();
^
cdp_tls.c:24:47: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_srv - 1].TLSMethod = TLS_server_method();
^
cdp_tls.c:27:45: warning: implicit declaration of function 'TLS_method' [-Wimplicit-function-declaration]
tls_methods[TLS_USE_TLSv1 - 1].TLSMethod = TLS_method();
^
cdp_tls.c:27:43: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1 - 1].TLSMethod = TLS_method();
^
cdp_tls.c:31:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_1_cli - 1].TLSMethod = TLS_client_method();
^
cdp_tls.c:34:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_1_srv - 1].TLSMethod = TLS_server_method();
^
cdp_tls.c:37:45: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_1 - 1].TLSMethod = TLS_method();
^
cdp_tls.c:41:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_2_cli - 1].TLSMethod = TLS_client_method();
^
cdp_tls.c:44:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_2_srv - 1].TLSMethod = TLS_server_method();
^
cdp_tls.c:47:45: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_2 - 1].TLSMethod = TLS_method();
^
cdp_tls.c:51:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_3_cli - 1].TLSMethod = TLS_client_method();
^
cdp_tls.c:52:54: error: 'TLS1_3_VERSION' undeclared (first use in this function)
tls_methods[TLS_USE_TLSv1_3_cli - 1].TLSMethodMin = TLS1_3_VERSION;
^
cdp_tls.c:52:54: note: each undeclared identifier is reported only once for each function it appears in
cdp_tls.c:54:49: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_3_srv - 1].TLSMethod = TLS_server_method();
^
cdp_tls.c:57:45: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_3 - 1].TLSMethod = TLS_method();
^
cdp_tls.c:62:48: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_PLUS - 1].TLSMethod = TLS_method();
^
cdp_tls.c:65:50: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_1_PLUS - 1].TLSMethod = TLS_method();
^
cdp_tls.c:68:50: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_2_PLUS - 1].TLSMethod = TLS_method();
^
cdp_tls.c:71:50: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
tls_methods[TLS_USE_TLSv1_3_PLUS - 1].TLSMethod = TLS_method();
^
cdp_tls.c: In function 'init_ssl_ctx':
cdp_tls.c:114:4: warning: implicit declaration of function 'SSL_CTX_set_min_proto_version' [-Wimplicit-function-declaration]
SSL_CTX_set_min_proto_version(
^
cdp_tls.c:123:4: warning: implicit declaration of function 'SSL_CTX_set_max_proto_version' [-Wimplicit-function-declaration]
SSL_CTX_set_max_proto_version(
^
../../Makefile.rules:100: recipe for target 'cdp_tls.o' failed
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/commit/defae2551f8839c5397523f135293fb…
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/commit/defae2551f8839c5397523f135293fb2a4056704/127741439(a)github.com>
### Description
In python (using module app_python3), in the xhttp module event callback (ksr_xhttp_event):
```
KSR.xhttp.xhttp_reply(400, "Bad Request", "", "")
return 1
```
When this code is processed, Kamailio logs the following errors:
```
6(247453) ERROR: <core> [core/parser/parse_fline.c:272]: parse_first_line(): parse_first_line: bad message (offset: 16)
6(247453) ERROR: <core> [core/parser/msg_parser.c:749]: parse_msg(): ERROR: parse_msg: message=<HTTP/1.1 400 Bad Request
Sia: SIP/2.0/TCP 127.0.0.1:41706
Server: kamailio (5.7.1 (x86_64/linux))
Content-Length: 0
>
6(247453) ERROR: <core> [core/msg_translator.c:3256]: build_sip_msg_from_buf(): parsing failed
```
(The correct HTTP reply is sent back to the client, so the issue is only with these logs.)
<br>
In kamailio script, in event_route[xhttp:request]:
```
xhttp_reply("400", "Bad Request", "", "");
exit;
```
In this case, there are no errors in Kamailio logs.
The two should be equivalent, but when using python we get errors when trying to send a specific HTTP reply.
<br>
When using python, if calling "KSR.jsonrpcs.dispatch()" for an unknown method, we get the same errors in Kamailio logs:
```
6(253511) ERROR: <core> [core/parser/parse_fline.c:272]: parse_first_line(): parse_first_line: bad message (offset: 19)
6(253511) ERROR: <core> [core/parser/msg_parser.c:749]: parse_msg(): ERROR: parse_msg: message=<HTTP/1.1 500 Method Not Found
Sia: SIP/2.0/TCP 127.0.0.1:59048
Content-Type: application/json
Server: kamailio (5.7.1 (x86_64/linux))
Content-Length: 84
{
"jsonrpc": "2.0",
"error": {
"code": 500,
"message": "Method Not Found"
}
}>
6(253511) ERROR: <core> [core/msg_translator.c:3256]: build_sip_msg_from_buf(): parsing failed
```
When the method is correct (client receives a HTTP 200), there are no errors in Kamailio logs.
<br>
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.7.1 (x86_64/linux) 4238e4
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, MEM_JOIN_FREE, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES, TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 4238e4
compiled on 16:52:41 Aug 24 2023 with gcc 8.5.0
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/3549
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/3549(a)github.com>