You can view, comment on, or merge this pull request online at:
https://github.com/kamailio/kamailio/pull/1685
-- Commit Summary --
* nsq: deprecate json and pua funcs for json api and pua_json modules
-- File Changes --
M src/modules/nsq/Makefile (3)
M src/modules/nsq/doc/nsq_admin.xml (131)
D src/modules/nsq/nsq_json.c (308)
D src/modules/nsq/nsq_json.h (59)
M src/modules/nsq/nsq_mod.c (96)
M src/modules/nsq/nsq_mod.h (23)
D src/modules/nsq/nsq_pua.c (537)
D src/modules/nsq/nsq_pua.h (34)
M src/modules/nsq/nsq_reader.c (18)
M src/modules/nsq/nsq_reader.h (3)
D src/modules/nsq/nsq_trans.c (452)
D src/modules/nsq/nsq_trans.h (42)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/1685.patchhttps://github.com/kamailio/kamailio/pull/1685.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/pull/1685
Pseudo variable $TV(un) - microseconds (not cached) wasn't working. Calling gettimeofday function fixes it.
<!-- Kamailio Pull Request Template -->
<!--
IMPORTANT:
- for detailed contributing guidelines, read:
https://github.com/kamailio/kamailio/blob/master/.github/CONTRIBUTING.md
- pull requests must be done to master branch, unless they are backports
of fixes from master branch to a stable branch
- backports to stable branches must be done with 'git cherry-pick -x ...'
- code is contributed under BSD for core and main components (tm, sl, auth, tls)
- code is contributed GPLv2 or a compatible license for the other components
- GPL code is contributed with OpenSSL licensing exception
-->
#### Pre-Submission Checklist
<!-- Go over all points below, and after creating the PR, tick all the checkboxes that apply -->
<!-- All points should be verified, otherwise, read the CONTRIBUTING guidelines from above-->
<!-- If you're unsure about any of these, don't hesitate to ask on sr-dev mailing list -->
- [X ] Commit message has the format required by CONTRIBUTING guide
- [X ] Commits are split per component (core, individual modules, libs, utils, ...)
- [X ] Each component has a single commit (if not, squash them into one commit)
- [X ] No commits to README files for modules (changes must be done to docbook files
in `doc/` subfolder, the README file is autogenerated)
#### Type Of Change
- [X ] Small bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds new functionality)
- [ ] Breaking change (fix or feature that would change existing functionality)
#### Checklist:
<!-- Go over all points below, and after creating the PR, tick the checkboxes that apply -->
- [X ] PR should be backported to stable branches
- [X ] Tested changes locally
- [ ] Related to issue #XXXX (replace XXXX with an open issue number)
#### Description
<!-- Describe your changes in detail -->
Pseudo variable $TV(un) - microseconds (not cached) wasn't working. Calling gettimeofday function fixes it.
You can view, comment on, or merge this pull request online at:
https://github.com/kamailio/kamailio/pull/1691
-- Commit Summary --
* pv: pv_time minor fix
-- File Changes --
M src/modules/pv/pv_time.c (5)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/1691.patchhttps://github.com/kamailio/kamailio/pull/1691.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/pull/1691
### Description
Kamailio dies complaining about a bad char in some header. I can say that I just updated from the nightly-devel apt repo, this problem never happened in latest v5.1 release.
### Troubleshooting
#### Reproduction
I don't know how to reproduce yet.
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
[BT-FULL_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/kama…
[BT-FULL_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/kama…
[INFO-LOCALS_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/…
[INFO-LOCALS_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/…
[LIST_core.kamailio.3329.1540415468.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3330.1540415471.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3331.1540415467.txt](https://github.com/kamailio/kamaili…
[LIST_core.kamailio.3333.1540415469.txt](https://github.com/kamailio/kamaili…
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by Kamailio, add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
The relevant log lines are:
```
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]: parse_addr_spec(): unexpected char [<] in status 6: ["910609864" <sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]: get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]: parse_headers(): bad header field [To: "910609864" <sip:;:929<>64(a)138.99.136.3>;tag=as664d068c#015#012Call-ID: 032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]: parse_addr_spec(): unexpected char [<] in status 6: ["910609864" <sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]: get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]: parse_headers(): bad header field [To: "910609864" <sip:;:929<>64(a)138.99.136.3>;tag=as664d068c#015#012Call-ID: 032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]: parse_addr_spec(): unexpected char [<] in status 6: ["910609864" <sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]: get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]: parse_headers(): bad header field [To: "910609864" <sip:;:929<>64(a)138.99.136.3>;tag=as664d068c#015#012Call-ID: 032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: pv [pv_core.c:1892]: pv_get_hdr(): error parsing headers
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/parse_addr_spec.c:718]: parse_addr_spec(): unexpected char [<] in status 6: ["910609864" <sip:;:929] .
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:164]: get_hdr_field(): bad to header
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: <core> [core/parser/msg_parser.c:337]: parse_headers(): bad header field [To: "910609864" <sip:;:929<>64(a)138.99.136.3>;tag=as664d068c#015#012Call-ID: 032f010653fed9170365045a4e1002]
Oct 24 16:11:11 sbc01 sbc[3330]: ERROR: pv [pv_core.c:704]: pv_get_callid(): cannot parse Call-Id header
```
#### SIP Traffic
<!--
If the issue is exposed by processing specific SIP messages, grab them with ngrep or save in a pcap file, then add them next, or attach to issue, or provide a link to download them (e.g., to a pastebin site).
-->
I have enabled a mirror server to capture all traffic, once the problem happens again I will be able to update this issue with SIP traffic.
### Possible Solutions
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
root@sbc01:/var/tmp# kamailio -v
version: kamailio 5.2.0-pre1 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
compiled with gcc 6.3.0
root@sbc01:/var/tmp#
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu 16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `uname -a`)
-->
```
root@sbc01:/var/tmp# uname -a
Linux sbc01.vozelia.com.pa 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64 GNU/Linux
root@sbc01:/var/tmp#
root@sbc01:/var/tmp# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.5 (stretch)
Release: 9.5
Codename: stretch
root@sbc01:/var/tmp#
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1687
### Description
Kamailio pipelimit module causes high CPU load (20 - 30 %) on servers with > 48 GB RAM even if Kamailio is idle (no active calls).
strace shows that every second `open /pro/stat` is called even if timer_interval is set to 10 seconds
`modparam("pipelimit", "timer_interval", 10)`
`1539343630.677932 open("/proc/stat", O_RDONLY) = 6`
`1539343630.678278 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6`
`1539343630.678418 open("/proc/net/udp", O_RDONLY) = 6`
`1539343630.679448 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.722522 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.753095 open("/proc/net/udp", O_RDONLY) = 6`
`1539343630.753838 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343630.783710 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.677966 open("/proc/stat", O_RDONLY) = 6`
`1539343631.678378 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 6`
`1539343631.678534 open("/proc/net/udp", O_RDONLY) = 6`
`1539343631.679784 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.717329 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.760763 open("/proc/net/udp", O_RDONLY) = 6`
`1539343631.761786 open("/proc/net/tcp", O_RDONLY) = 6`
`1539343631.805426 open("/proc/net/tcp", O_RDONLY) = 6`
strace shows that with 8 GB RAM the read syscall takes 100us (average) - with 48GB it takes 586us (average)!
see 5 seconds interval strace for 48GB:
`2018-09-21 07:55:38,201`
`strace: Process 6907 attached`
`strace: Process 6907 detached`
`% time seconds usecs/call calls errors syscall`
`------ ----------- ----------- --------- --------- ----------------`
`99.79 1.259694 586 2148 read`
` 0.06 0.000761 19 41 open`
` 0.04 0.000469 8 61 61 pause`
` 0.04 0.000462 13 36 munmap`
` 0.03 0.000336 8 41 close`
` 0.02 0.000257 7 36 mmap`
` 0.02 0.000234 3 80 61 rt_sigreturn`
` 0.01 0.000113 3 36 fstat`
` 0.00 0.000017 3 5 kill`
`------ ----------- ----------- --------- --------- ----------------`
`100.00 1.262343 2484 122 total`
`2018-09-21 07:55:43,225`
In the documentation (https://kamailio.org/docs/modules/5.1.x/modules/pipelimit.html) I found the following:
> As reading the CPU load average is relatively expensive (opening /proc/stat, parsing it, etc), this only happens once every timer_interval seconds and consequently the FEEDBACK value is only at these intervals recomputed. This in turn makes it difficult for the drop rate to adjust quickly. Worst case scenarios are request rates going up/down instantly by thousands - it takes up to 20 seconds for the controller to adapt to the new request rate.
IMHO the doc is misleading because the periodic timer used for this is hardcoded with 1000ms.
see
`( 349 pipelimit.c ) timer_add(pl_timer, MS_TO_TICKS(1000)); /* Start it after 1000ms */`
Conclusion:
Pipelimit polls periodically every second /proc/stats and /proc/net/udp|tcp. The modparam timer_interval does not affect the timer period.
On servers with >48 GB this causes high CPU load.
#### Reproduction
always reproducible.
### Possible Solutions
Use modparam timer_interval to initialize the `pl_timer` interval
### Additional Information
version: kamailio 5.1.2 (x86_64/linux)
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: unknown
* **Operating System**: Centos 7.4
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1675
Continued from issue:
https://github.com/kamailio/kamailio/issues/1583
Say you have a kamailio config file with variables ordered as A(int),B(int),C(string),D(int) – (all belonging to the same group).
Kamailio first creates a linked list of these variables in a given group and calculates the group size as follows [actual code](https://github.com/kamailio/kamailio/blob/master/src/core/cfg/cfg_scr…):
```
Pseudo-code
---------------------------------------------------------------------------------------------
group->size = 0
group->vars = null // the pointer to the head of the linked list of variables
create_group_linked_list:
for variable in group:
if variable is integer:
round up group_size to the nearest 4 byte multiple
group_size += sizeof(int)
if variable is string:
round up group_size to the nearest 8 byte multiple
group_size += sizeof(str) //str is a kamailio type (size 16 in our C++ compiler)
if group->vars is null:
group->vars = variable
else
variables->next = group->vars
group->vars = variable
---------------------------------------------------------------------------------------------
Linked list creation
NULL
group->size 0
A(int)
group->size 4
B(int) -> A
group->size 8
C(string) -> B -> A
group->size 24
D(int) -> C -> B -> A
group->size 28
```
However, this linked list is not used as is. it is copied to a single shared memory block for all child processes.
This is done in the [cfg_shmize()](https://github.com/kamailio/kamailio/blob/master/src/core/cfg… which further calls [cfg_script_fixup()](https://github.com/kamailio/kamailio/blob/master/src/co…
Basically, the function goes through the linked list of variables created above and calculates the offset of each variable in the memory block assuming that the offset of the first variable is 0.
Following is the pseudo-code of what cfg_script_fixup() does for the linked list and offset calculation:
```
Pseudo-code
--------------------------------------------------------------------------------------------
offset = 0
node = group->vars:
while node != null:
if node's value is int:
round up offset to nearest multiple of 4
node->offset = offset
offset += sizeof(int)
if node's value is str:
round up offset to nearest multiple of 8
node->offset = offset
offset += sizeof(str)
node = node -> next
--------------------------------------------------------------------------------------------
linked list (from above):
D(int) -> C(string) -> B(int) -> A(int)
offset=0
D
D->offset = 0
offset = 4
C
C->offset = 8
offset = 24
B
B->offset = 24
offset = 28
A
A->offset = 28
offset = 32 (which would be the actual group size)
```
So, as visible, **the order in which the linked list is being processed on the two cases is causing issues** (from A to D in one case and from D to A in the next case).
while the group size calculation code block leads to a group size of 28, the group size according to the second code block would be 32 and the offset of the last variable would we 28. This would lead to the last variable (all 4 bytes) getting corrupted.
My temporary solution is to recalculate the group size every time a variable is added in the first code block by re-traversing the linked list but is definitely not the ideal solution.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/1679
Module: kamailio
Branch: master
Commit: 2ecf601c472bb81b9cf4ffd5b1ac17c4dfd742f2
URL: https://github.com/kamailio/kamailio/commit/2ecf601c472bb81b9cf4ffd5b1ac17c…
Author: mtirpak <miklos.tirpak(a)gmail.com>
Committer: mtirpak <miklos.tirpak(a)gmail.com>
Date: 2018-10-25T13:57:17+02:00
core: variables declared in the config file could cause memory corruption
The config variables that are declared in the config file were recorded
in the reverse order as their padding was calculated, which could cause
the allocated memory block to be smaller as required at the end.
Credits go to vinesinha.
---
Modified: src/core/cfg/cfg_script.c
---
Diff: https://github.com/kamailio/kamailio/commit/2ecf601c472bb81b9cf4ffd5b1ac17c…
Patch: https://github.com/kamailio/kamailio/commit/2ecf601c472bb81b9cf4ffd5b1ac17c…
---
diff --git a/src/core/cfg/cfg_script.c b/src/core/cfg/cfg_script.c
index b85d103696..9873f70405 100644
--- a/src/core/cfg/cfg_script.c
+++ b/src/core/cfg/cfg_script.c
@@ -35,7 +35,7 @@ cfg_script_var_t *new_cfg_script_var(char *gname, char *vname, unsigned int type
char *descr)
{
cfg_group_t *group;
- cfg_script_var_t *var;
+ cfg_script_var_t *var, **last_var;
int gname_len, vname_len, descr_len;
LM_DBG("declaring %s.%s\n", gname, vname);
@@ -112,9 +112,15 @@ cfg_script_var_t *new_cfg_script_var(char *gname, char *vname, unsigned int type
memset(var, 0, sizeof(cfg_script_var_t));
var->type = type;
- /* add the variable to the group */
- var->next = (cfg_script_var_t *)(void *)group->vars;
- group->vars = (char *)(void *)var;
+ /* Add the variable to the end of the group.
+ * The order is important because the padding depends on that.
+ * The list will be travelled later again, which must be done in
+ * the same order. */
+ last_var = (cfg_script_var_t **)(void **)&group->vars;
+ while ((*last_var))
+ last_var = &((*last_var)->next);
+ *last_var = var;
+ var->next = NULL;
/* clone the name of the variable */
var->name = (char *)pkg_malloc(sizeof(char) * (vname_len + 1));
@@ -282,6 +288,14 @@ int cfg_script_fixup(cfg_group_t *group, unsigned char *block)
}
}
+ /* Sanity check for the group size, make sure that the
+ * newly calculated size equals the already calculated
+ * group size. */
+ if (offset != group->size) {
+ LM_ERR("BUG: incorrect group size: %d; previously calculated value: %d \n", offset, group->size);
+ goto error;
+ }
+
/* allocate a handle even if it will not be used to
directly access the variable, like handle->variable
cfg_get_* functions access the memory block via the handle
It was assigning the length instead of comparing it.
<!-- Kamailio Pull Request Template -->
<!--
IMPORTANT:
- for detailed contributing guidelines, read:
https://github.com/kamailio/kamailio/blob/master/.github/CONTRIBUTING.md
- pull requests must be done to master branch, unless they are backports
of fixes from master branch to a stable branch
- backports to stable branches must be done with 'git cherry-pick -x ...'
- code is contributed under BSD for core and main components (tm, sl, auth, tls)
- code is contributed GPLv2 or a compatible license for the other components
- GPL code is contributed with OpenSSL licensing exception
-->
#### Pre-Submission Checklist
<!-- Go over all points below, and after creating the PR, tick all the checkboxes that apply -->
<!-- All points should be verified, otherwise, read the CONTRIBUTING guidelines from above-->
<!-- If you're unsure about any of these, don't hesitate to ask on sr-dev mailing list -->
- [X ] Commit message has the format required by CONTRIBUTING guide
- [X] Commits are split per component (core, individual modules, libs, utils, ...)
- [X] Each component has a single commit (if not, squash them into one commit)
- [X ] No commits to README files for modules (changes must be done to docbook files
in `doc/` subfolder, the README file is autogenerated)
#### Type Of Change
- [ X] Small bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds new functionality)
- [ ] Breaking change (fix or feature that would change existing functionality)
#### Checklist:
<!-- Go over all points below, and after creating the PR, tick the checkboxes that apply -->
- [] PR should be backported to stable branches
- [X] Tested changes locally
- [ ] Related to issue #XXXX (replace XXXX with an open issue number)
#### Description
<!-- Describe your changes in detail -->
The tag length was being assigning instead of being compared.
You can view, comment on, or merge this pull request online at:
https://github.com/kamailio/kamailio/pull/1688
-- Commit Summary --
* evapi: minor fix in tag length compare
-- File Changes --
M src/modules/evapi/evapi_dispatch.c (2)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/1688.patchhttps://github.com/kamailio/kamailio/pull/1688.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/pull/1688