some modules can have child processes that each maintain a database connection. the number of child processes for a module can sometimes be set using a modparam() for that module

On Mon, Jul 10, 2017 at 4:23 PM, Alex Balashov <abalashov@evaristesys.com> wrote:
Hi,

By way of illustration, I have one server with children=8, a single UDP
listener, and 23 processes total:

--
21117   attendant
21118   udp receiver child=0 sock=xxx.xxx.xxx.xxx:5060
21119   udp receiver child=1 sock=xxx.xxx.xxx.xxx:5060
21120   udp receiver child=2 sock=xxx.xxx.xxx.xxx:5060
21121   udp receiver child=3 sock=xxx.xxx.xxx.xxx:5060
21122   udp receiver child=4 sock=xxx.xxx.xxx.xxx:5060
21123   udp receiver child=5 sock=xxx.xxx.xxx.xxx:5060
21124   udp receiver child=6 sock=xxx.xxx.xxx.xxx:5060
21125   udp receiver child=7 sock=xxx.xxx.xxx.xxx:5060
21126   slow timer
21127   timer
21128   MI FIFO
21129   ctl handler
21130   RTIMER USEC EXEC
21131   RTIMER USEC EXEC
21132   RTIMER USEC EXEC
21133   RTIMER USEC EXEC
21134   RTIMER USEC EXEC
21135   RTIMER USEC EXEC
21136   RTIMER USEC EXEC
21137   Dialog KA Timer
21138   Dialog Clean Timer
21139   tcp main process

# kamcmd -s /tmp/kamailio_ctl ps | wc -l
23
--

This appears to result in 23 total database connections to the Postgres
server:

--
# lsof -n | grep -i '^kam' | grep IPv4 | grep TCP | grep -i :postgres | wc -l
23
--

But I have another Kamailio server that has three listeners, two UDP and
one TCP, and children=24, i.e.

--
children=24
listen=udp:xxx.xxx.xxx.xxx:5060
listen=udp:yyy.yyy.yyy.yyy:5060
listen=tcp:zzz.zzz.zzz.zzz:5060
--

I am given to understand that the total number of worker processes there
will be (children_setting * listeners), i.e. 72 in this case, plus some
number of rtimer and supervisory processes, as in my case. And true
enough:

--
# kamcmd -s /tmp/kamailio_ctl ps | wc -l
86
# kamcmd -s /tmp/kamailio_ctl ps | sed 's/65.254.44.195/yyy.yyy.yyy.yyy/g' | sed 's/65.254.44.194/zzz.zzz.zzz.zzz/g'
22065   main process - attendant
22069   udp receiver child=0 sock=zzz.zzz.zzz.zzz:5060
22070   udp receiver child=1 sock=zzz.zzz.zzz.zzz:5060
22071   udp receiver child=2 sock=zzz.zzz.zzz.zzz:5060
22073   udp receiver child=3 sock=zzz.zzz.zzz.zzz:5060
22074   udp receiver child=4 sock=zzz.zzz.zzz.zzz:5060
22076   udp receiver child=5 sock=zzz.zzz.zzz.zzz:5060
22079   udp receiver child=6 sock=zzz.zzz.zzz.zzz:5060
22080   udp receiver child=7 sock=zzz.zzz.zzz.zzz:5060
22082   udp receiver child=8 sock=zzz.zzz.zzz.zzz:5060
22085   udp receiver child=9 sock=zzz.zzz.zzz.zzz:5060
22086   udp receiver child=10 sock=zzz.zzz.zzz.zzz:5060
22089   udp receiver child=11 sock=zzz.zzz.zzz.zzz:5060
22091   udp receiver child=12 sock=zzz.zzz.zzz.zzz:5060
22093   udp receiver child=13 sock=zzz.zzz.zzz.zzz:5060
22095   udp receiver child=14 sock=zzz.zzz.zzz.zzz:5060
22097   udp receiver child=15 sock=zzz.zzz.zzz.zzz:5060
22099   udp receiver child=16 sock=zzz.zzz.zzz.zzz:5060
22100   udp receiver child=17 sock=zzz.zzz.zzz.zzz:5060
22103   udp receiver child=18 sock=zzz.zzz.zzz.zzz:5060
22105   udp receiver child=19 sock=zzz.zzz.zzz.zzz:5060
22107   udp receiver child=20 sock=zzz.zzz.zzz.zzz:5060
22109   udp receiver child=21 sock=zzz.zzz.zzz.zzz:5060
22111   udp receiver child=22 sock=zzz.zzz.zzz.zzz:5060
22113   udp receiver child=23 sock=zzz.zzz.zzz.zzz:5060
22115   udp receiver child=0 sock=yyy.yyy.yyy.yyy:5060
22117   udp receiver child=1 sock=yyy.yyy.yyy.yyy:5060
22119   udp receiver child=2 sock=yyy.yyy.yyy.yyy:5060
22121   udp receiver child=3 sock=yyy.yyy.yyy.yyy:5060
22123   udp receiver child=4 sock=yyy.yyy.yyy.yyy:5060
22125   udp receiver child=5 sock=yyy.yyy.yyy.yyy:5060
22127   udp receiver child=6 sock=yyy.yyy.yyy.yyy:5060
22129   udp receiver child=7 sock=yyy.yyy.yyy.yyy:5060
22131   udp receiver child=8 sock=yyy.yyy.yyy.yyy:5060
22132   udp receiver child=9 sock=yyy.yyy.yyy.yyy:5060
22135   udp receiver child=10 sock=yyy.yyy.yyy.yyy:5060
22137   udp receiver child=11 sock=yyy.yyy.yyy.yyy:5060
22139   udp receiver child=12 sock=yyy.yyy.yyy.yyy:5060
22141   udp receiver child=13 sock=yyy.yyy.yyy.yyy:5060
22143   udp receiver child=14 sock=yyy.yyy.yyy.yyy:5060
22145   udp receiver child=15 sock=yyy.yyy.yyy.yyy:5060
22147   udp receiver child=16 sock=yyy.yyy.yyy.yyy:5060
22149   udp receiver child=17 sock=yyy.yyy.yyy.yyy:5060
22151   udp receiver child=18 sock=yyy.yyy.yyy.yyy:5060
22153   udp receiver child=19 sock=yyy.yyy.yyy.yyy:5060
22155   udp receiver child=20 sock=yyy.yyy.yyy.yyy:5060
22156   udp receiver child=21 sock=yyy.yyy.yyy.yyy:5060
22159   udp receiver child=22 sock=yyy.yyy.yyy.yyy:5060
22161   udp receiver child=23 sock=yyy.yyy.yyy.yyy:5060
22163   slow timer
22165   timer
22167   secondary timer
22168   ctl handler
22169   RTIMER USEC EXEC
22170   RTIMER USEC EXEC
22172   RTIMER USEC EXEC
22173   RTIMER USEC EXEC
22175   Dialog Clean Timer
22176   TIMER NH
22177   TIMER NH
22178   TIMER NH
22179   tcp receiver (generic) child=0
22180   tcp receiver (generic) child=1
22182   tcp receiver (generic) child=2
22184   tcp receiver (generic) child=3
22186   tcp receiver (generic) child=4
22188   tcp receiver (generic) child=5
22190   tcp receiver (generic) child=6
22192   tcp receiver (generic) child=7
22194   tcp receiver (generic) child=8
22196   tcp receiver (generic) child=9
22198   tcp receiver (generic) child=10
22200   tcp receiver (generic) child=11
22202   tcp receiver (generic) child=12
22204   tcp receiver (generic) child=13
22206   tcp receiver (generic) child=14
22208   tcp receiver (generic) child=15
22210   tcp receiver (generic) child=16
22212   tcp receiver (generic) child=17
22214   tcp receiver (generic) child=18
22216   tcp receiver (generic) child=19
22218   tcp receiver (generic) child=20
22220   tcp receiver (generic) child=21
22223   tcp receiver (generic) child=22
22225   tcp receiver (generic) child=23
22227   tcp main process
--

But, for some reason, the number of database connections to Postgres
seems to be considerably higher than the number of total processes:

--
# lsof -n | grep '^kam' | grep TCP | grep IPv4 | grep ':postgres' | wc -l
114
--

This is not due to orphaned/unterminated Kamailio processes. There's
only one process lineage:

--
        |-kamailio(22065)-+-kamailio(22069)
        |                 |-kamailio(22070)
        |                 |-kamailio(22071)
        |                 |-kamailio(22073)
        |                 |-kamailio(22074)
        |                 |-kamailio(22076)
        |                 |-kamailio(22079)
        |                 |-kamailio(22080)
        |                 |-kamailio(22082)
        |                 |-kamailio(22085)
        |                 |-kamailio(22086)
        |                 |-kamailio(22089)
        |                 |-kamailio(22091)
        |                 |-kamailio(22093)
        |                 |-kamailio(22095)
        |                 |-kamailio(22097)
        |                 |-kamailio(22099)
        |                 |-kamailio(22100)
        |                 |-kamailio(22103)
        |                 |-kamailio(22105)
        |                 |-kamailio(22107)
        |                 |-kamailio(22109)
        |                 |-kamailio(22111)
        |                 |-kamailio(22113)
        |                 |-kamailio(22115)
        |                 |-kamailio(22117)
        |                 |-kamailio(22119)
        |                 |-kamailio(22121)
        |                 |-kamailio(22123)
        |                 |-kamailio(22125)
        |                 |-kamailio(22127)
        |                 |-kamailio(22129)
        |                 |-kamailio(22131)
        |                 |-kamailio(22132)
        |                 |-kamailio(22135)
        |                 |-kamailio(22137)
        |                 |-kamailio(22139)
        |                 |-kamailio(22141)
        |                 |-kamailio(22143)
        |                 |-kamailio(22145)
        |                 |-kamailio(22147)
        |                 |-kamailio(22149)
        |                 |-kamailio(22151)
        |                 |-kamailio(22153)
        |                 |-kamailio(22155)
        |                 |-kamailio(22156)
        |                 |-kamailio(22159)
        |                 |-kamailio(22161)
        |                 |-kamailio(22163)
        |                 |-kamailio(22165)
        |                 |-kamailio(22167)
        |                 |-kamailio(22168)
        |                 |-kamailio(22169)
        |                 |-kamailio(22170)
        |                 |-kamailio(22172)
        |                 |-kamailio(22173)
        |                 |-kamailio(22175)
        |                 |-kamailio(22176)
        |                 |-kamailio(22177)
        |                 |-kamailio(22178)
        |                 |-kamailio(22179)
        |                 |-kamailio(22180)
        |                 |-kamailio(22182)
        |                 |-kamailio(22184)
        |                 |-kamailio(22186)
        |                 |-kamailio(22188)
        |                 |-kamailio(22190)
        |                 |-kamailio(22192)
        |                 |-kamailio(22194)
        |                 |-kamailio(22196)
        |                 |-kamailio(22198)
        |                 |-kamailio(22200)
        |                 |-kamailio(22202)
        |                 |-kamailio(22204)
        |                 |-kamailio(22206)
        |                 |-kamailio(22208)
        |                 |-kamailio(22210)
        |                 |-kamailio(22212)
        |                 |-kamailio(22214)
        |                 |-kamailio(22216)
        |                 |-kamailio(22218)
        |                 |-kamailio(22220)
        |                 |-kamailio(22223)
        |                 |-kamailio(22225)
        |                 `-kamailio(22227)
--

Yet, for some reason, some of the processes have two established DB
connections from different ports:

--
kamailio  22220        root    3u     IPv4          189895303        0t0        TCP 127.0.0.1:46550->127.0.0.1:postgres (ESTABLISHED)
kamailio  22220        root   86u     IPv4          189895067        0t0        TCP 127.0.0.1:46458->127.0.0.1:postgres (ESTABLISHED)
kamailio  22223        root    3u     IPv4          189895315        0t0        TCP 127.0.0.1:46556->127.0.0.1:postgres (ESTABLISHED)
kamailio  22223        root   86u     IPv4          189895067        0t0        TCP 127.0.0.1:46458->127.0.0.1:postgres (ESTABLISHED)
kamailio  22225        root    3u     IPv4          189895321        0t0        TCP 127.0.0.1:46560->127.0.0.1:postgres (ESTABLISHED)
kamailio  22225        root   86u     IPv4          189895067        0t0        TCP 127.0.0.1:46458->127.0.0.1:postgres (ESTABLISHED)
kamailio  22227        root   86u     IPv4          189895067        0t0        TCP 127.0.0.1:46458->127.0.0.1:postgres (ESTABLISHED)
--

... while others don't ...

--
kamailio  22111        root    3u     IPv4          189894820        0t0        TCP 127.0.0.1:46326->127.0.0.1:postgres (ESTABLISHED)
kamailio  22113        root    3u     IPv4          189894828        0t0        TCP 127.0.0.1:46332->127.0.0.1:postgres (ESTABLISHED)
kamailio  22115        root    3u     IPv4          189894834        0t0        TCP 127.0.0.1:46334->127.0.0.1:postgres (ESTABLISHED)
kamailio  22117        root    3u     IPv4          189894842        0t0        TCP 127.0.0.1:46338->127.0.0.1:postgres (ESTABLISHED)
kamailio  22119        root    3u     IPv4          189894848        0t0        TCP 127.0.0.1:46340->127.0.0.1:postgres (ESTABLISHED)
--

So, this gives rise to two questions in my mind:

1. I don't seem to understand the relationship between the number of
database connection handles and the total number of child processes.

I had always assumed that only SIP workers and other processes
specialised into a role with possible "database involvement" (e.g.
rtimer, async workers, etc.) would hold database connections.

2. What's going on in the second scenario such that some processes have
two connections?

Both servers running:

--
# kamcmd -s /tmp/kamailio_ctl core.version
kamailio 4.4.6 (x86_64/linux) becbde
--

Insights much appreciated!

-- Alex

--
Alex Balashov | Principal | Evariste Systems LLC

Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/

_______________________________________________
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users