I'm testing failure scenarios before going to production.

This test has the MySQL server offline to observe the Proxy's behavior when it cannot reach the DB server.

In this case we are using the DB for checking a global black list, we are not using the DB for auth or location.

If the DB server goes offline after Kamailio has started there does not seem to be an issue. However if the DB server is offline when Kamailio is starting, Kamailio fails to start correctly.

Kamailio log (hostnames and IP addresses changed or redacted):

``` Jun 4 10:21:24 robo-proxy kamailio: INFO: <core> [tcp_main.c:4748]: init_tcp(): using epoll_lt as the io watch method (auto detected) Jun 4 10:21:24 robo-proxy kamailio: WARNING: <core> [daemonize.c:360]: daemonize(): pid file contains old pid, replacing pid Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: rr [../outbound/api.h:54]: ob_load_api(): Failed to import bind_ob Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: rr [rr_mod.c:160]: mod_init(): outbound module not available Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: pike [pike.c:115]: pike_init(): PIKE - initializing Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: pike [ip_tree.c:88]: init_lock_set(): probing 256 set size Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:175]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:225]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:175]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [udp_server.c:225]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 1, pid 4635, "udp receiver child=0 sock=127.0.0.1:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4635]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 3, pid 4638, "udp receiver child=2 sock=127.0.0.1:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4638]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 2, pid 4636, "udp receiver child=1 sock=127.0.0.1:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4636]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 4, pid 4640, "udp receiver child=3 sock=127.0.0.1:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4640]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 8, pid 4647, "udp receiver child=3 sock=1.2.3.4:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4647]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 7, pid 4645, "udp receiver child=2 sock=1.2.3.4:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 5, pid 4642, "udp receiver child=0 sock=1.2.3.4:5060" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4645]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4642]: CRITICAL: <core> [main.c:1621]: main_loop(): Cannot fork Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4654]: ERROR: <core> [pt.c:341]: fork_process(): init_child failed for process 10, pid 4654, "timer" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4635 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4636 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4638 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4640 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4642 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4645 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: ALERT: <core> [main.c:781]: handle_sigs(): child process 4647 exited normally, status=255 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4634]: INFO: <core> [main.c:799]: handle_sigs(): terminating due to SIGCHLD Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4653]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4660]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4644]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4661]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 12 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 13 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 14 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 15 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 16 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 17 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4658]: INFO: <core> [main.c:850]: sig_usr(): signal 15 received Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 18 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 19 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 20 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 21 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 26 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 24 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 23 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [pt.c:479]: fork_tcp_process(): init_child failed for process 13, pid 4659, "tcp receiver (generic) child=0" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4659]: ERROR: <core> [tcp_main.c:4871]: tcp_init_children(): fork failed: Connection refused Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: db_mysql [km_my_con.c:111]: db_mysql_new_connection(): driver error: Can't connect to MySQL server on 'lab-mysql01' (111) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [db.c:322]: db_do_init2(): could not add connection to the pool Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: sqlops [sql_api.c:166]: sql_connect(): failed to connect to the database [nomo_con] Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [sr_module.c:923]: init_mod_child(): Error while initializing module sqlops (/usr/lib/x86_64-linux-gnu/kamailio/modules/sqlops.so) Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [pt.c:479]: fork_tcp_process(): init_child failed for process 16, pid 4664, "tcp receiver (generic) child=3" Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4664]: ERROR: <core> [tcp_main.c:4871]: tcp_init_children(): fork failed: Connection refused Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 5 Jun 4 10:21:24 robo-proxy /usr/sbin/kamailio[4666]: CRITICAL: <core> [pass_fd.c:293]: receive_fd(): EOF on 28 ```

--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/192