We use the kamailio with dmq. One Kamailio gets an SIP Register and process it and forwards it though DMQ to another kamailio. here it should be saved in location as well.
During saving this location the kamailio crashes. For me (not an dev) it seems to crash during getting the length of the AOR. This is not happening every time while kamailio gets an DMQ register but we can reproduce it very easy. (at least this is the line pointing to referred in the backtrace)
the SIP DMQ Message looks like
Oct 20 14:44:53 messi /usr/sbin/kamailio[15059]: NOTICE: <script>: Got KDMQ Message, handle it.. KDMQ sip:usrloc@217.116.120.39:5060 SIP/2.0 Via: SIP/2.0/UDP 172.20.21.3;branch=z9hG4bK7031.da526d01000000000000000000000000.0 To: sip:usrloc@217.116.120.39:5060 From: sip:usrloc@172.20.21.3:5060;tag=c67e1a50505355c50500c05660eca619-6820 CSeq: 10 KDMQ Call-ID: 77bf92d7284e3548-25608@172.20.21.3 Content-Length: 430 Max-Forwards: 0 Content-Type: application/json
{"action":2,"aor":"1016811e7@sipgate.de","ruid":"uloc-562103eb-6411-a4a1","c":"sip:1016811e7@XX.XXX.XXX.XX:3072;line=2u4rbxb1","received":"","path":"sip:XXX.XXX.XXX.XXX;lr","callid":"3134343130393735323137323138-pbswhv8xostq","user_agent":"snom870/8.7.5.13","instance":"urn:uuid:f3495943-cd1f-4de0-8d9c-000413413E79","expires":10,"cseq":28786,"flags":0,"cflags":0,"q":1000,"last_modified":1445345060,"methods":6111,"reg_id":1}
we managed to get an core dump and it says:
GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/sbin/kamailio...Reading symbols from /usr/lib/debug/.build-id/b9/e50c34b5b5de6da1e2473514911ce96a8a094d.debug...done. done. [New LWP 15069]
warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/kamailio -f /etc/kamailio/kamailio_sip_location_proxy.cfg -P /var/run'. Program terminated with signal 11, Segmentation fault. #0 0x00007f573d8c7770 in db_load_urecord_by_ruid (_c=0x7f573f616278, _d=_d@entry=0x7f5736ef5498, _ruid=_ruid@entry=0x7f573ae6d200) at udomain.c:822 822 udomain.c: No such file or directory. (gdb) bt #0 0x00007f573d8c7770 in db_load_urecord_by_ruid (_c=0x7f573f616278, _d=_d@entry=0x7f5736ef5498, _ruid=_ruid@entry=0x7f573ae6d200) at udomain.c:822 #1 0x00007f573d8cd0d5 in get_urecord_by_ruid (_d=0x7f5736ef5498, _aorhash=2416148423, _ruid=_ruid@entry=0x7f573ae6d200, _r=_r@entry=0x7ffc358958a0, _c=_c@entry=0x7ffc358958b0) at udomain.c:1177 #2 0x00007f573ac6720a in delete_contact (aor=..., ci=<optimized out>) at usrloc_sync.c:109 #3 usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffc35895a00, node=0x7f5736f9e6d8) at usrloc_sync.c:394 #4 0x00007f573ae8b10b in worker_loop (id=id@entry=1) at worker.c:105 #5 0x00007f573ae721e8 in child_init (rank=<optimized out>) at dmq.c:293 #6 child_init (rank=<optimized out>) at dmq.c:279 #7 0x000000000056d4f8 in init_mod_child (m=0x7f573f5fdf20, rank=rank@entry=0) at sr_module.c:915 #8 0x000000000056d442 in init_mod_child (m=0x7f573f5fe418, rank=rank@entry=0) at sr_module.c:912 #9 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 #10 0x00000000004aa78c in main_loop () at main.c:1650 #11 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 (gdb) bt full #0 0x00007f573d8c7770 in db_load_urecord_by_ruid (_c=0x7f573f616278, _d=_d@entry=0x7f5736ef5498, _ruid=_ruid@entry=0x7f573ae6d200) at udomain.c:822 ci = 0x7f573daeaf00 columns = {0x7f573daeac30, 0x7f573daeac20, 0x7f573daeac10, 0x7f573daeac00, 0x7f573daeabf0, 0x7f573daeabe0, 0x7f573daeabd0, 0x7f573daeabc0, 0x7f573daeabb0, 0x7f573daeaba0, 0x7f573daeab90, 0x7f573daeab80, 0x7f573daeab50, 0x7f573daeac60, 0x7f573daeab70, 0x7f573daeab60, 0x7f573daeab40, 0x7f573daeab30, 0x7f573daeab20, 0x7f573daeac50, 0x7f573daeac40} keys = {0x7f573daeac60} order = <optimized out> vals = {{type = DB1_STR, nul = 0, free = 2, val = {int_val = 40974704, ll_val = 40974704, double_val = 2.0244193594913908e-316, time_val = 40974704, string_val = 0x2713970 "uloc-562103eb-6411-a4a1", str_val = {s = 0x2713970 "uloc-562103eb-6411-a4a1", len = 23}, blob_val = {s = 0x2713970 "uloc-562103eb-6411-a4a1", len = 23}, bitmap_val = 40974704}}} res = 0x7f573f6164e8 row = <optimized out> contact = {s = 0x272edd0 "sip:1016811e7@XX.XXX.XXX.XX:3072;line=e52c50xi", len = 46} aor = {s = 0x37fc50 <Address 0x37fc50 out of bounds>, len = 898193808} aorbuf = '\000' <repeats 511 times> domain = <optimized out> r = 0x0 c = <optimized out> __FUNCTION__ = "db_load_urecord_by_ruid" #1 0x00007f573d8cd0d5 in get_urecord_by_ruid (_d=0x7f5736ef5498, _aorhash=2416148423, _ruid=_ruid@entry=0x7f573ae6d200, _r=_r@entry=0x7ffc358958a0, _c=_c@entry=0x7ffc358958b0) at udomain.c:1177 sl = 16327 i = <optimized out> r = <optimized out> c = <optimized out> #2 0x00007f573ac6720a in delete_contact (aor=..., ci=<optimized out>) at usrloc_sync.c:109 c = <optimized out> ci = 0x7f573ae6d200 _d = 0x7f5736ef5498 r = <optimized out> #3 usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffc35895a00, node=0x7f5736f9e6d8) at usrloc_sync.c:394 content_length = <optimized out> body = <optimized out> jdoc = {root = 0x27137b0, flags = 0, buf = { s = 0x7f5736fd6662 "{"action":2,"aor":"1016811e7@sipgate.de","ruid":"uloc-562103eb-6411-a4a1","c":"sip:1016811e7@XX.XXX.XXX.XX:3072;line=2u4rbxb1","received":"","path":"sip:217.116.120.247;lr","callid":"313434313039373"..., len = 438}, malloc_fn = 0x7f573fa1ba00 <malloc>, free_fn = 0x7f573fa1b920 <free>} it = <optimized out> ci = {ruid = {s = 0x2713970 "uloc-562103eb-6411-a4a1", len = 23}, c = 0x7ffc358958d0, received = {s = 0x2719190 "", len = 0}, path = 0x7ffc358958e0, expires = 1445345660, q = 1000, callid = 0x7ffc358958f0, cseq = 28786, flags = 2, cflags = 0, user_agent = 0x7ffc35895900, sock = 0x0, methods = 6111, instance = { s = 0x27193d0 "urn:uuid:f3495943-cd1f-4de0-8d9c-000413413E79", len = 47}, reg_id = 1, server_id = 0, tcpconn_id = -1, keepalive = 0, xavp = 0x0, last_modified = 1445345060} action = <optimized out> expires = <optimized out> cseq = <optimized out> flags = <optimized out> cflags = <optimized out> q = <optimized out> last_modified = <optimized out> methods = <optimized out> reg_id = <optimized out> aor = {s = <optimized out>, len = 20} ruid = <optimized out> c = {s = 0x27190e0 "sip:1016811e7@IPADDRESS:3072;line=2u4rbxb1", len = 46} received = <optimized out> path = {s = 0x2713a00 "sip:IPADDRESS;lr", len = 24} callid = {s = 0x2719290 "3134343130393735323137323138-pbswhv8xostq", len = 41} user_agent = {s = 0x2719340 "snom870/8.7.5.13", len = 16} instance = <optimized out> __FUNCTION__ = "usrloc_dmq_handle_msg" #4 0x00007f573ae8b10b in worker_loop (id=id@entry=1) at worker.c:105 worker = 0x7f5736f957f0 current_job = 0x7f5736fa0a00 peer_response = {resp_code = 0, content_type = {s = 0x0, len = 0}, reason = {s = 0x0, len = 0}, body = {s = 0x0, len = 0}} ret_value = <optimized out> not_parsed = 0 dmq_node = 0x7f5736f9e6d8 __FUNCTION__ = "worker_loop" #5 0x00007f573ae721e8 in child_init (rank=<optimized out>) at dmq.c:293 ---Type <return> to continue, or q <return> to quit--- i = <optimized out> newpid = <optimized out> #6 child_init (rank=<optimized out>) at dmq.c:279 No locals. #7 0x000000000056d4f8 in init_mod_child (m=0x7f573f5fdf20, rank=rank@entry=0) at sr_module.c:915 __FUNCTION__ = "init_mod_child" #8 0x000000000056d442 in init_mod_child (m=0x7f573f5fe418, rank=rank@entry=0) at sr_module.c:912 __FUNCTION__ = "init_mod_child" #9 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 No locals. #10 0x00000000004aa78c in main_loop () at main.c:1650 i = <optimized out> pid = <optimized out> si = <optimized out> si_desc = "udp receiver child=3 sock=217.116.120.39:5060\000\000\000\020\213k\000\000\000\000\000\252\311k\000\000\000\000\000\003\000\000\000\000\000\000\000(E\311\066W\177\000\000\001\000\000\000\000\000\000\000\000P:\251\313M\a\342\004\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\250\222\241\000\000\000\000\000\240\222\241\000\000\000\000" nrprocs = <optimized out> __FUNCTION__ = "main_loop" #11 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7ffc35896efe "" tmp_len = 0 port = 1 proto = 1 options = 0x6baf50 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:" ret = -1 seed = 1198920520 rfd = 4 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = <optimized out> n_lst = 0x0 p = <optimized out> st = {st_dev = 14, st_ino = 30859, st_nlink = 2, st_mode = 16877, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = { tv_sec = 1444903317, tv_nsec = 438957686}, st_mtim = {tv_sec = 1445345013, tv_nsec = 829833044}, st_ctim = {tv_sec = 1445345020, tv_nsec = 417829857}, __unused = {0, 0, 0}} __FUNCTION__ = "main" (gdb)
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373
Can you show the definition of table location? Is the username column a varchar and has not-NULL constraint?
Also, check to see if there is a record with ruid=uloc-562103eb-6411-a4a1 and if yes, what is the value for username column?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149573800
Hi Daniel,
the location looks like:
CREATE TABLE `location` ( `id` int(10) unsigned NOT NULL AUTO_INCREMENT, `ruid` varchar(64) NOT NULL DEFAULT '', `username` varchar(64) NOT NULL DEFAULT '', `domain` varchar(64) DEFAULT NULL, `contact` varchar(255) NOT NULL DEFAULT '', `received` varchar(128) DEFAULT NULL, `path` varchar(512) DEFAULT NULL, `expires` datetime NOT NULL DEFAULT '2030-05-28 21:32:15', `q` float(10,2) NOT NULL DEFAULT '1.00', `callid` varchar(255) NOT NULL DEFAULT 'Default-Call-ID', `cseq` int(11) NOT NULL DEFAULT '1', `last_modified` datetime NOT NULL DEFAULT '1900-01-01 00:00:01', `flags` int(11) NOT NULL DEFAULT '0', `cflags` int(11) NOT NULL DEFAULT '0', `user_agent` varchar(255) NOT NULL DEFAULT '', `socket` varchar(64) DEFAULT NULL, `methods` int(11) DEFAULT NULL, `instance` varchar(255) DEFAULT NULL, `reg_id` int(11) NOT NULL DEFAULT '0', `server_id` int(11) NOT NULL DEFAULT '0', `connection_id` int(11) NOT NULL DEFAULT '0', `keepalive` int(11) NOT NULL DEFAULT '0', `partition` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `ruid_idx` (`ruid`)
and yes there is an entry for this ruid with username "1016811e7"
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149574672
Can you get the output in gdb, frame 0, for:
``` p (ROW_VALUES(RES_ROWS(res)))[19] ```
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149575605
Hi Daniel,
we are not sure how to get this value. I tryed to select frame 0 and just paste the line you provided. this is what happend:
(gdb) select-frame 0 (gdb) p (ROW_VALUES(RES_ROWS(res)))[19] No symbol "ROW_VALUES" in current context. (gdb)
i guess this is not what you need. Can you give us more specific instructions? Would it help we provide the backtrace it self? it's about 80Mbyte
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149583281
Try:
``` p res->rows->values[19] ```
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149584093
Hi Daniel,
there we go...
(gdb) p res->rows->values[19] $1 = {type = 576, nul = 0, free = 0, val = {int_val = 3669072, ll_val = 3669072, double_val = 1.8127624273180341e-317, time_val = 3669072, string_val = 0x37fc50 <Address 0x37fc50 out of bounds>, str_val = {s = 0x37fc50 <Address 0x37fc50 out of bounds>, len = 1063076752}, blob_val = {s = 0x37fc50 <Address 0x37fc50 out of bounds>, len = 1063076752}, bitmap_val = 3669072}} (gdb)
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-149789900
So that value seems invalid.
Looking at the code I see that this is done during a delete contact operation. Is your usrloc configured with db only mode? Because is doing query to database, not using the cache.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-150264844
yes it is. The purpose of that kamailio is not having sip call traffic but only save the location in DB.
modparam("usrloc", "db_mode", 3)
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-150496756
Can you try with latest 4.3 branch? I pushed a patch for this issue.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-150563267
We tried it, but it still crashes. This time:
``` GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/sbin/kamailio...Reading symbols from /usr/lib/debug/.build-id/81/2f0de7537c5e5ef4c282e12de0c29ee7ea15f0.debug...done. done. [New LWP 11903]
warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/kamailio -f /etc/kamailio/kamailio.cfg -P /var/run/kamailio/kamailio.'. Program terminated with signal 11, Segmentation fault. #0 free_urecord (_r=0x2050b70) at urecord.c:94 94 urecord.c: No such file or directory. (gdb) bt #0 free_urecord (_r=0x2050b70) at urecord.c:94 #1 0x00007fa5861ebb33 in usrloc_get_all_ucontact (node=node@entry=0x7fa58251b990) at usrloc_sync.c:214 #2 0x00007fa5861eceb6 in usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffd3312d4e0, node=0x7fa58251b990) at usrloc_sync.c:398 #3 0x00007fa58641110b in worker_loop (id=id@entry=0) at worker.c:105 #4 0x00007fa5863f81e8 in child_init (rank=<optimized out>) at dmq.c:293 #5 child_init (rank=<optimized out>) at dmq.c:279 #6 0x000000000056d4f8 in init_mod_child (m=0x7fa58ab84f20, rank=rank@entry=0) at sr_module.c:915 #7 0x000000000056d442 in init_mod_child (m=0x7fa58ab85418, rank=rank@entry=0) at sr_module.c:912 #8 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 #9 0x00000000004aa78c in main_loop () at main.c:1650 #10 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 (gdb) bt full #0 free_urecord (_r=0x2050b70) at urecord.c:94 ptr = 0x3 __FUNCTION__ = "free_urecord" #1 0x00007fa5861ebb33 in usrloc_get_all_ucontact (node=node@entry=0x7fa58251b990) at usrloc_sync.c:214 rval = <optimized out> len = <optimized out> buf = 0x7fa58aba7400 cp = 0x7fa58aba747d ruid = {s = 0x7fa58aba7462 "uloc-559b8fc0-3fa8-20b9", len = 23} aorhash = <optimized out> send_sock = <optimized out> flags = <optimized out> aor = {s = 0x0, len = 0} r = 0x2050b70 _d = 0x7fa58247b498 ptr = 0x0 res = <optimized out> __FUNCTION__ = "usrloc_get_all_ucontact" #2 0x00007fa5861eceb6 in usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffd3312d4e0, node=0x7fa58251b990) at usrloc_sync.c:398 content_length = <optimized out> body = <optimized out> jdoc = {root = 0x2050b20, flags = 0, buf = {s = 0x7fa582526a85 "{"action":3}", len = 12}, malloc_fn = 0x7fa58afa2a00 <malloc>, free_fn = 0x7fa58afa2920 <free>} it = <optimized out> ci = {ruid = {s = 0x0, len = 0}, c = 0x7ffd3312d3b0, received = {s = 0x0, len = 0}, path = 0x7ffd3312d3c0, expires = 0, q = 0, callid = 0x7ffd3312d3d0, cseq = 0, flags = 2, cflags = 0, user_agent = 0x7ffd3312d3e0, sock = 0x0, methods = 0, instance = {s = 0x0, len = 0}, reg_id = 0, server_id = 0, tcpconn_id = -1, keepalive = 0, xavp = 0x0, last_modified = 0} action = <optimized out> expires = <optimized out> cseq = <optimized out> flags = <optimized out> cflags = <optimized out> q = <optimized out> last_modified = <optimized out> methods = <optimized out> reg_id = <optimized out> aor = {s = <optimized out>, len = 0} ruid = <optimized out> c = {s = 0x0, len = 0} received = <optimized out> path = {s = 0x0, len = 0} callid = {s = 0x0, len = 0} user_agent = {s = 0x0, len = 0} instance = <optimized out> __FUNCTION__ = "usrloc_dmq_handle_msg" #3 0x00007fa58641110b in worker_loop (id=id@entry=0) at worker.c:105 worker = 0x7fa58251b7d8 current_job = 0x7fa582527270 peer_response = {resp_code = 0, content_type = {s = 0x0, len = 0}, reason = {s = 0x0, len = 0}, body = {s = 0x0, len = 0}} ret_value = <optimized out> not_parsed = 0 dmq_node = 0x7fa58251b990 __FUNCTION__ = "worker_loop" #4 0x00007fa5863f81e8 in child_init (rank=<optimized out>) at dmq.c:293 i = <optimized out> newpid = <optimized out> #5 child_init (rank=<optimized out>) at dmq.c:279 No locals. #6 0x000000000056d4f8 in init_mod_child (m=0x7fa58ab84f20, rank=rank@entry=0) at sr_module.c:915 __FUNCTION__ = "init_mod_child" #7 0x000000000056d442 in init_mod_child (m=0x7fa58ab85418, rank=rank@entry=0) at sr_module.c:912 __FUNCTION__ = "init_mod_child" #8 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 No locals. #9 0x00000000004aa78c in main_loop () at main.c:1650 i = <optimized out> pid = <optimized out> si = <optimized out> si_desc = "udp receiver child=3 sock=217.116.120.39:5060\000\000\000\f\b\000\000\000\000\000\000 9l\000\000\000\000\000\003\000\000\000\000\000\000\000(\245!\202\245\177\000\000\001\000\000\000\000\000\000\000\000\322\266\217\037\372 <\004\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\250\222\241\000\000\000\000\000\240\222\241\000\000\000\000" nrprocs = <optimized out> __FUNCTION__ = "main_loop" #10 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7ffd3312fcca "" tmp_len = 0 port = 1 proto = 1 options = 0x6baf50 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:" ret = -1 seed = 1431985423 rfd = 4 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = <optimized out> n_lst = 0x0 p = <optimized out> st = {st_dev = 14, st_ino = 30859, st_nlink = 2, st_mode = 16877, st_uid = 109, st_gid = 301, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1444903317, tv_nsec = 438957686}, st_mtim = {tv_sec = 1445519056, tv_nsec = 19559617}, st_ctim = {tv_sec = 1445607933, tv_nsec = 740608953}, __unused = {0, 0, 0}} __FUNCTION__ = "main" ```
Does that help?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-150579455
Looking at the sources I think there was a mistake interpreting a return code in dmq_usrloc -- can you try with patch 13cde2bb1deda9cc67925ca01910e41150c4435e (it is only in master)?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-150614571
After adapting the patch (a git cherry-pick would not work) it still crashes. I hope I did it right, here is the new backtrace:
``` GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/sbin/kamailio...Reading symbols from /usr/lib/debug/.build-id/78/f34ac70f52f40e085a6b521ebbf282fe4b468d.debug...done. done. [New LWP 29787]
warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/sbin/kamailio -f /etc/kamailio/kamailio_sip_location_proxy.cfg -P /var/run'. Program terminated with signal 11, Segmentation fault. #0 usrloc_get_all_ucontact (node=node@entry=0x7fdea80a7588) at usrloc_sync.c:203 203 usrloc_sync.c: No such file or directory.
Thread 1 (Thread 0x7fdeb3066700 (LWP 29787)): #0 usrloc_get_all_ucontact (node=node@entry=0x7fdea80a7588) at usrloc_sync.c:203 #1 0x00007fdeabd6fe86 in usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffd776ced10, node=0x7fdea80a7588) at usrloc_sync.c:398 #2 0x00007fdeabf9410b in worker_loop (id=id@entry=1) at worker.c:105 #3 0x00007fdeabf7b1e8 in child_init (rank=<optimized out>) at dmq.c:293 #4 child_init (rank=<optimized out>) at dmq.c:279 #5 0x000000000056d4f8 in init_mod_child (m=0x7fdeb0707f20, rank=rank@entry=0) at sr_module.c:915 #6 0x000000000056d442 in init_mod_child (m=0x7fdeb0708418, rank=rank@entry=0) at sr_module.c:912 #7 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 #8 0x00000000004aa78c in main_loop () at main.c:1650 #9 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 #0 usrloc_get_all_ucontact (node=node@entry=0x7fdea80a7588) at usrloc_sync.c:203 rval = <optimized out> len = <optimized out> buf = 0x7fdeb0729310 cp = 0x7fdeb072938a ruid = {s = 0x7fdeb0729372 "uloc-56139a40-3acd-5", len = 20} aorhash = 0 send_sock = <optimized out> flags = 0 aor = {s = 0x7fdea80aab59 "{"action":3}", len = 12704544} r = 0x0 _d = 0x7fdea7ffe498 ptr = 0x0 res = -1 __FUNCTION__ = "usrloc_get_all_ucontact" #1 0x00007fdeabd6fe86 in usrloc_dmq_handle_msg (msg=<optimized out>, resp=0x7ffd776ced10, node=0x7fdea80a7588) at usrloc_sync.c:398 content_length = <optimized out> body = <optimized out> jdoc = {root = 0xc1db20, flags = 0, buf = {s = 0x7fdea80aab59 "{"action":3}", len = 12}, malloc_fn = 0x7fdeb2725a00 <malloc>, free_fn = 0x7fdeb2725920 <free>} it = <optimized out> ci = {ruid = {s = 0x0, len = 0}, c = 0x7ffd776cebe0, received = {s = 0x0, len = 0}, path = 0x7ffd776cebf0, expires = 0, q = 0, callid = 0x7ffd776cec00, cseq = 0, flags = 2, cflags = 0, user_agent = 0x7ffd776cec10, sock = 0x0, methods = 0, instance = {s = 0x0, len = 0}, reg_id = 0, server_id = 0, tcpconn_id = -1, keepalive = 0, xavp = 0x0, last_modified = 0} action = <optimized out> expires = <optimized out> cseq = <optimized out> flags = <optimized out> cflags = <optimized out> q = <optimized out> last_modified = <optimized out> methods = <optimized out> reg_id = <optimized out> aor = {s = <optimized out>, len = 0} ruid = <optimized out> c = {s = 0x0, len = 0} received = <optimized out> path = {s = 0x0, len = 0} callid = {s = 0x0, len = 0} user_agent = {s = 0x0, len = 0} instance = <optimized out> __FUNCTION__ = "usrloc_dmq_handle_msg" #2 0x00007fdeabf9410b in worker_loop (id=id@entry=1) at worker.c:105 worker = 0x7fdea809e7f0 current_job = 0x7fdea80a9170 peer_response = {resp_code = 0, content_type = {s = 0x0, len = 0}, reason = {s = 0x0, len = 0}, body = {s = 0x0, len = 0}} ret_value = <optimized out> not_parsed = 0 dmq_node = 0x7fdea80a7588 __FUNCTION__ = "worker_loop" #3 0x00007fdeabf7b1e8 in child_init (rank=<optimized out>) at dmq.c:293 i = <optimized out> newpid = <optimized out> #4 child_init (rank=<optimized out>) at dmq.c:279 No locals. #5 0x000000000056d4f8 in init_mod_child (m=0x7fdeb0707f20, rank=rank@entry=0) at sr_module.c:915 __FUNCTION__ = "init_mod_child" #6 0x000000000056d442 in init_mod_child (m=0x7fdeb0708418, rank=rank@entry=0) at sr_module.c:912 __FUNCTION__ = "init_mod_child" #7 0x000000000056e91b in init_child (rank=rank@entry=0) at sr_module.c:941 No locals. #8 0x00000000004aa78c in main_loop () at main.c:1650 i = <optimized out> pid = <optimized out> si = <optimized out> si_desc = "udp receiver child=3 sock=217.116.120.39:5060\000\000\000\f\b\000\000\000\000\000\000 9l\000\000\000\000\000\003\000\000\000\000\000\000\000(\325\331\247\336\177\000\000\001\000\000\000\000\000\000\000\000\032\323\n^xs\375\004\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\250\222\241\000\000\000\000\000\240\222\241\000\000\000\000" nrprocs = <optimized out> __FUNCTION__ = "main_loop" #9 0x0000000000427292 in main (argc=<optimized out>, argv=<optimized out>) at main.c:2547 cfg_stream = <optimized out> c = <optimized out> r = <optimized out> tmp = 0x7ffd776d096c "" tmp_len = 0 port = 1 proto = 1 options = 0x6baf50 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:" ret = -1 seed = 3607138207 rfd = 4 debug_save = <optimized out> debug_flag = <optimized out> dont_fork_cnt = <optimized out> n_lst = 0x0 p = <optimized out> st = {st_dev = 14, st_ino = 30859, st_nlink = 2, st_mode = 16877, st_uid = 109, st_gid = 301, __pad0 = 0, st_rdev = 0, st_size = 60, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1444903317, tv_nsec = 438957686}, st_mtim = {tv_sec = 1445853952, tv_nsec = 149421394}, st_ctim = {tv_sec = 1445853956, tv_nsec = 177425657}, __unused = {0, 0, 0}} __FUNCTION__ = "main" ```
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-151091947
Grab commit eeb3c160ecddd65aa76997ed98e9b1be8e08ab20 and try again.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-151280539
Thanks. Running now in our dev environment, no crash so far. I would attach a patch that works against the 4.3 branch, but Github won't allow me to attach .patch files (and no .txt neither, because I don't have write access to the repository).
So this is the plain text patch, that works for us:
``` index 40e89d3..15c0fb3 100644 --- a/modules/dmq_usrloc/usrloc_sync.c +++ b/modules/dmq_usrloc/usrloc_sync.c @@ -197,14 +197,14 @@ void usrloc_get_all_ucontact(dmq_node_t* node) memcpy( &aorhash, cp, sizeof(aorhash)); cp = (char*)cp + sizeof(aorhash);
+ r = 0; + ptr = 0; res = dmq_ul.get_urecord_by_ruid(_d, aorhash, &ruid, &r, &ptr); - aor = r->aor; - if (res > 0) { - LM_DBG("'%.*s' Not found in usrloc\n", aor.len, ZSW(aor.s)); - dmq_ul.release_urecord(r); - dmq_ul.unlock_udomain(_d, &aor); + if (res < 0) { + LM_DBG("'%.*s' Not found in usrloc\n", ruid.len, ZSW(ruid.s)); continue; } + aor = r->aor; LM_DBG("- AoR: %.*s AoRhash=%d Flags=%d\n", aor.len, aor.s, aorhash, flags);
while (ptr) { ```
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-151447184
I will backport the missing patches to 4.3.
Github doesn't allow attaching patches/files, as I understood -- they recommend making pull request instead of attaching patches (fork the repo, then push a patch to the clone and initiate a pull request from the clone: https://help.github.com/articles/creating-a-pull-request/ ).
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#issuecomment-151451326
Closed #373.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/issues/373#event-446611692
-
Daniel-Constantin Mierla -
Sebastian Damm
-
vinzens