This one seems to be a udp receiver and looks ok:
#0 0x00007f9958d359c3 in __recvfrom_nocancel () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a029443 in udp_rcv_loop () at core/udp_server.c:460
#2 0x000055ce69f969cf in main_loop () at main.c:1621
#3 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
It waits for packets from the network. Did you send registration via UDP?
Also the others seems to be rather ok, couple of them are specific timer
processes for modules ...
Cheers,
Daniel
On 24.12.18 13:22, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
Can you see how many kamailio processes are
running (w.g., with ps)? Are
there expected number of there?
Same number of processes when I start K with
proper rtpengine set and
with one that has an rtpengine that does not respond.
If yes, take the PID of few of them and attach
with gdb, then grab the
back trace in order to see what they do.
There is 41 processes. Below is a few
different ones.
-- Juha
---------------
#0 0x00007f9958d04210 in __pause_nocancel () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055ce69f98509 in main_loop () at main.c:1755
#2 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d359c3 in __recvfrom_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a029443 in udp_rcv_loop () at core/udp_server.c:460
#2 0x000055ce69f969cf in main_loop () at main.c:1621
#3 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958c7fe89 in __GI___sigwaitinfo (set=<optimized out>, info=0x0)
at ../sysdeps/unix/sysv/linux/sigwaitinfo.c:56
#1 0x000055ce6a04c36a in slow_timer_main () at core/timer.c:1093
#2 0x000055ce69f971ab in main_loop () at main.c:1677
#3 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d359c3 in __recvfrom_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a153168 in async_task_run (idx=1) at core/async_task.c:269
#2 0x000055ce6a151f9c in async_task_child_init (rank=0) at core/async_task.c:185
#3 0x000055ce6a0d5e3a in init_child (rank=0) at core/sr_module.c:867
#4 0x000055ce69f977cc in main_loop () at main.c:1703
#5 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d350a3 in __epoll_wait_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x00007f99554117e7 in io_wait_loop_epoll (h=0x7f9955643340 <io_h>, t=10,
repeat=0) at ../../core/io_wait.h:1034
#2 0x00007f9955415d1a in io_listen_loop (fd_no=1, cs_lst=0x55ce6c6b6680) at
io_listener.c:281
#3 0x00007f9955431c4c in mod_child (rank=0) at ctl.c:338
#4 0x000055ce6a0d5a84 in init_mod_child (m=0x7f99580b0ea0, rank=0) at
core/sr_module.c:843
#5 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b12d0, rank=0) at
core/sr_module.c:839
#6 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b1600, rank=0) at
core/sr_module.c:839
#7 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b1ac0, rank=0) at
core/sr_module.c:839
#8 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b1ee0, rank=0) at
core/sr_module.c:839
#9 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b2720, rank=0) at
core/sr_module.c:839
#10 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b2a30, rank=0) at
core/sr_module.c:839
#11 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b3210, rank=0) at
core/sr_module.c:839
#12 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b34a0, rank=0) at
core/sr_module.c:839
#13 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b36f0, rank=0) at
core/sr_module.c:839
#14 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b3a20, rank=0) at
core/sr_module.c:839
#15 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b4270, rank=0) at
core/sr_module.c:839
#16 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b44c0, rank=0) at
core/sr_module.c:839
#17 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b47f0, rank=0) at
core/sr_module.c:839
#18 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b4c80, rank=0) at
core/sr_module.c:839
#19 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b5250, rank=0) at
core/sr_module.c:839
#20 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b6e20, rank=0) at
core/sr_module.c:839
#21 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7490, rank=0) at
core/sr_module.c:839
#22 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b77a0, rank=0) at
core/sr_module.c:839
#23 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7b20, rank=0) at
core/sr_module.c:839
#24 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7d70, rank=0) at
core/sr_module.c:839
#25 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b87d0, rank=0) at
core/sr_module.c:839
#26 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b8f50, rank=0) at
core/sr_module.c:839
#27 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b95d0, rank=0) at
core/sr_module.c:839
#28 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580be830, rank=0) at
core/sr_module.c:839
#29 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580beeb0, rank=0) at
core/sr_module.c:839
#30 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf220, rank=0) at
core/sr_module.c:839
#31 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf5c0, rank=0) at
core/sr_module.c:839
#32 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf810, rank=0) at
core/sr_module.c:839
#33 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bfea0, rank=0) at
core/sr_module.c:839
#34 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c01a0, rank=0) at
core/sr_module.c:839
#35 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0580, rank=0) at
core/sr_module.c:839
#36 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0f90, rank=0) at
core/sr_module.c:839
#37 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c14c0, rank=0) at
core/sr_module.c:839
#38 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c17d0, rank=0) at
core/sr_module.c:839
#39 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1d90, rank=0) at
core/sr_module.c:839
#40 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1fe0, rank=0) at
core/sr_module.c:839
#41 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2310, rank=0) at
core/sr_module.c:839
#42 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2cf0, rank=0) at
core/sr_module.c:839
#43 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3340, rank=0) at
core/sr_module.c:839
#44 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3700, rank=0) at
core/sr_module.c:839
#45 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4590, rank=0) at
core/sr_module.c:839
#46 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c47b0, rank=0) at
core/sr_module.c:839
#47 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c49d0, rank=0) at
core/sr_module.c:839
#48 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4c90, rank=0) at
core/sr_module.c:839
#49 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5230, rank=0) at
core/sr_module.c:839
#50 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5530, rank=0) at
core/sr_module.c:839
#51 0x000055ce6a0d5e59 in init_child (rank=0) at core/sr_module.c:870
#52 0x000055ce69f977cc in main_loop () at main.c:1703
#53 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d2d3a3 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a185727 in sleep_us (nusecs=100) at core/ut.h:513
#2 0x000055ce6a185898 in fork_basic_utimer (child_id=-1, desc=0x7f994f19434a
"SIPDUMP WRITE TIMER", make_sock=1,
f=0x7f994f192e3c <sipdump_timer_exec>, param=0x0, uinterval=100) at
core/timer_proc.c:116
#3 0x00007f994f18e782 in child_init (rank=0) at sipdump_mod.c:126
#4 0x000055ce6a0d5a84 in init_mod_child (m=0x7f99580b4270, rank=0) at
core/sr_module.c:843
#5 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b44c0, rank=0) at
core/sr_module.c:839
#6 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b47f0, rank=0) at
core/sr_module.c:839
#7 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b4c80, rank=0) at
core/sr_module.c:839
#8 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b5250, rank=0) at
core/sr_module.c:839
#9 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b6e20, rank=0) at
core/sr_module.c:839
#10 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7490, rank=0) at
core/sr_module.c:839
#11 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b77a0, rank=0) at
core/sr_module.c:839
#12 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7b20, rank=0) at
core/sr_module.c:839
#13 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b7d70, rank=0) at
core/sr_module.c:839
#14 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b87d0, rank=0) at
core/sr_module.c:839
#15 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b8f50, rank=0) at
core/sr_module.c:839
#16 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580b95d0, rank=0) at
core/sr_module.c:839
#17 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580be830, rank=0) at
core/sr_module.c:839
#18 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580beeb0, rank=0) at
core/sr_module.c:839
#19 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf220, rank=0) at
core/sr_module.c:839
#20 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf5c0, rank=0) at
core/sr_module.c:839
#21 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf810, rank=0) at
core/sr_module.c:839
#22 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bfea0, rank=0) at
core/sr_module.c:839
#23 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c01a0, rank=0) at
core/sr_module.c:839
#24 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0580, rank=0) at
core/sr_module.c:839
#25 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0f90, rank=0) at
core/sr_module.c:839
#26 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c14c0, rank=0) at
core/sr_module.c:839
#27 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c17d0, rank=0) at
core/sr_module.c:839
#28 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1d90, rank=0) at
core/sr_module.c:839
#29 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1fe0, rank=0) at
core/sr_module.c:839
#30 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2310, rank=0) at
core/sr_module.c:839
#31 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2cf0, rank=0) at
core/sr_module.c:839
#32 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3340, rank=0) at
core/sr_module.c:839
#33 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3700, rank=0) at
core/sr_module.c:839
#34 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4590, rank=0) at
core/sr_module.c:839
#35 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c47b0, rank=0) at
core/sr_module.c:839
#36 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c49d0, rank=0) at
core/sr_module.c:839
#37 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4c90, rank=0) at
core/sr_module.c:839
#38 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5230, rank=0) at
core/sr_module.c:839
#39 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5530, rank=0) at
core/sr_module.c:839
#40 0x000055ce6a0d5e59 in init_child (rank=0) at core/sr_module.c:870
#41 0x000055ce69f977cc in main_loop () at main.c:1703
#42 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d2d3a3 in __select_nocancel () at ../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a185727 in sleep_us (nusecs=60000000) at core/ut.h:513
#2 0x000055ce6a185a8b in fork_sync_timer (child_id=-1, desc=0x7f994466af8a "USRLOC
Timer", make_sock=1,
f=0x7f9944646573 <ul_local_timer>, param=0x0, interval=60000) at
core/timer_proc.c:221
#3 0x00007f994464538a in child_init (_rank=0) at usrloc_mod.c:398
#4 0x000055ce6a0d5a84 in init_mod_child (m=0x7f99580bf5c0, rank=0) at
core/sr_module.c:843
#5 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bf810, rank=0) at
core/sr_module.c:839
#6 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580bfea0, rank=0) at
core/sr_module.c:839
#7 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c01a0, rank=0) at
core/sr_module.c:839
#8 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0580, rank=0) at
core/sr_module.c:839
#9 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c0f90, rank=0) at
core/sr_module.c:839
#10 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c14c0, rank=0) at
core/sr_module.c:839
#11 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c17d0, rank=0) at
core/sr_module.c:839
#12 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1d90, rank=0) at
core/sr_module.c:839
#13 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c1fe0, rank=0) at
core/sr_module.c:839
#14 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2310, rank=0) at
core/sr_module.c:839
#15 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c2cf0, rank=0) at
core/sr_module.c:839
#16 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3340, rank=0) at
core/sr_module.c:839
#17 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c3700, rank=0) at
core/sr_module.c:839
#18 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4590, rank=0) at
core/sr_module.c:839
#19 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c47b0, rank=0) at
core/sr_module.c:839
#20 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c49d0, rank=0) at
core/sr_module.c:839
#21 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4c90, rank=0) at
core/sr_module.c:839
#22 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5230, rank=0) at
core/sr_module.c:839
#23 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5530, rank=0) at
core/sr_module.c:839
#24 0x000055ce6a0d5e59 in init_child (rank=0) at core/sr_module.c:870
#25 0x000055ce69f977cc in main_loop () at main.c:1703
#26 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d04270 in __nanosleep_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x00007f9958d041da in __sleep (seconds=0) at ../sysdeps/posix/sleep.c:55
#2 0x000055ce6a1857d1 in fork_basic_timer (child_id=-1, desc=0x7ffd8be61540 "RTIMER
EXEC child=0 timer=minute_timer",
make_sock=1, f=0x7f994275d35a <stm_timer_exec>, param=0x7f99580d0190,
interval=60) at core/timer_proc.c:77
#3 0x00007f994275cfd6 in child_init (rank=0) at rtimer_mod.c:166
#4 0x000055ce6a0d5a84 in init_mod_child (m=0x7f99580c4590, rank=0) at
core/sr_module.c:843
#5 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c47b0, rank=0) at
core/sr_module.c:839
#6 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c49d0, rank=0) at
core/sr_module.c:839
#7 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c4c90, rank=0) at
core/sr_module.c:839
#8 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5230, rank=0) at
core/sr_module.c:839
#9 0x000055ce6a0d5707 in init_mod_child (m=0x7f99580c5530, rank=0) at
core/sr_module.c:839
#10 0x000055ce6a0d5e59 in init_child (rank=0) at core/sr_module.c:870
#11 0x000055ce69f977cc in main_loop () at main.c:1703
#12 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d350a3 in __epoll_wait_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a1d2713 in io_wait_loop_epoll (h=0x55ce6a653fc0 <io_w>, t=2,
repeat=0) at core/io_wait.h:1034
#2 0x000055ce6a1e77bb in tcp_receive_loop (unix_sock=45) at core/tcp_read.c:1974
#3 0x000055ce6a09ae90 in tcp_init_children () at core/tcp_main.c:4853
#4 0x000055ce69f97ab2 in main_loop () at main.c:1711
#5 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
#0 0x00007f9958d350a3 in __epoll_wait_nocancel () at
../sysdeps/unix/syscall-template.S:84
#1 0x000055ce6a05f47a in io_wait_loop_epoll (h=0x55ce6a60adc0 <io_h>, t=5,
repeat=0) at core/io_wait.h:1034
#2 0x000055ce6a0971d1 in tcp_main_loop () at core/tcp_main.c:4547
#3 0x000055ce69f97e0c in main_loop () at main.c:1719
#4 0x000055ce69f9eb39 in main (argc=17, argv=0x7ffd8be61d18) at main.c:2645
Kamailio Advanced Training - Mar 4-6, 2019 in Berlin; Mar 25-27, 2019, in Washington, DC,
USA --