i just noticed that my proxy had crashed on invite request from attacker:
May 14 22:03:06 sars /usr/sbin/sip-proxy[10932]: INFO: INVITE tel:004412127921\ 94 by untrusted sip:210.125.64.233 from <210.125.64.233> May 14 22:03:06 sars /usr/sbin/sip-proxy[10932]: : <core> [mem/q_malloc.c:159]:\ qm_debug_frag(): BUG: qm_*: prev. fragm. tail overwritten(c0c0c000, abcdefed)[\ 0xb70f6a64:0xb70f6a7c]! May 14 22:03:08 sars /usr/sbin/sip-proxy[11014]: : <core> [pass_fd.c:293]: rece\ ive_fd(): ERROR: receive_fd: EOF on 24 May 14 22:03:08 sars /usr/sbin/sip-proxy[10913]: ALERT: <core> [main.c:775]: ha\ ndle_sigs(): child process 10932 exited by a signal 6 May 14 22:03:08 sars /usr/sbin/sip-proxy[10913]: ALERT: <core> [main.c:778]: ha\ ndle_sigs(): core was generated
Program terminated with signal 6, Aborted. #0 0xb7782424 in __kernel_vsyscall () (gdb) where #0 0xb7782424 in __kernel_vsyscall () #1 0xb7616941 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #2 0xb7619d72 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #3 0x08179f86 in qm_debug_frag (qm=0xb6dea008, f=0xb70f6a64) at mem/q_malloc.c:161 #4 0x0817ac3a in qm_malloc (qm=0xb6dea008, size=48, file=0x81f3169 "<core>: action.c", func=0x81f42f0 "do_action", line=780) at mem/q_malloc.c:386 #5 0x0805e798 in do_action (h=0xbffc1ca0, a=0xbffc1d48, msg=0xb72c6928) at action.c:780 #6 0xb1c5ac1d in pv_set_ruri (msg=0xb72c6928, param=0xb6f75630, op=254, val=0xbffc1e0c) at pv_core.c:2019 #7 0xb1b5df59 in tel2sip (_msg=0xb72c6928, _uri=0xb6f75c70 "H\207\367\266\004", _hostpart=0xb6f75530 "\254y\367\266\004", _res=0xb6f75624 "\006") at checks.c:405 #8 0x0805fdf7 in do_action (h=0xbffc21e0, a=0xb6f77858, msg=0xb72c6928) at action.c:1117 #9 0x08067293 in run_actions (h=0xbffc21e0, a=0xb6f77858, msg=0xb72c6928) at action.c:1599 #10 0x080678e2 in run_actions_safe (h=0xbffc39ac, a=0xb6f77858, msg=0xb72c6928) at action.c:1664 #11 0x081015fe in rval_get_int (h=0xbffc39ac, msg=0xb72c6928, i=0xbffc2528, rv=0xb6f779fc, cache=0x0) at rvalue.c:924 #12 0x08103f83 in rval_expr_eval_int (h=0xbffc39ac, msg=0xb72c6928, res=0xbffc2528, rve=0xb6f779f8) at rvalue.c:1918 #13 0x0810416e in rval_expr_eval_int (h=0xbffc39ac, msg=0xb72c6928, res=0xbffc27c4, rve=0xb6f78360) at rvalue.c:1926 #14 0x0805fa26 in do_action (h=0xbffc39ac, a=0xb6f78820, msg=0xb72c6928) at action.c:1075 #15 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f78820, msg=0xb72c6928) at action.c:1599 #16 0x0805fca0 in do_action (h=0xbffc39ac, a=0xb6f788c4, msg=0xb72c6928) at action.c:1094 ---Type <return> to continue, or q <return> to quit--- #17 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f788c4, msg=0xb72c6928) at action.c:1599 #18 0x0805fc5f in do_action (h=0xbffc39ac, a=0xb6f78968, msg=0xb72c6928) at action.c:1090 #19 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f78968, msg=0xb72c6928) at action.c:1599 #20 0x0805e00d in do_action (h=0xbffc39ac, a=0xb6e7720c, msg=0xb72c6928) at action.c:715 #21 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e5161c, msg=0xb72c6928) at action.c:1599 #22 0x0805e00d in do_action (h=0xbffc39ac, a=0xb6e50238, msg=0xb72c6928) at action.c:715 #23 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e50238, msg=0xb72c6928) at action.c:1599 #24 0x0805fc5f in do_action (h=0xbffc39ac, a=0xb6e50bfc, msg=0xb72c6928) at action.c:1090 #25 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e4891c, msg=0xb72c6928) at action.c:1599 #26 0x0806797a in run_top_route (a=0xb6e4891c, msg=0xb72c6928, c=0x0) at action.c:1685 #27 0x080e2bcf in receive_msg ( buf=0x82f99e0 "INVITE tel:00441212792194 SIP/2.0\r\nVia: SIP/2.0/UDP 210.125.64.233;branch=z9hG4bK4KmbLm4c\r\nMax-Forwards: 69\r\nFrom: sip:210.125.64.233;tag=qua2A5c8s9VJZ\r\nTo: tel:00441212792194\r\nContact: <sip:210.1"..., len=1115, rcv_info=0xbffc3bb0) at receive.c:211 #28 0x081702cd in udp_rcv_loop () at udp_server.c:536 #29 0x080ad9a0 in main_loop () at main.c:1617 #30 0x080b098f in main (argc=17, argv=0xbffc3e64) at main.c:2533
perhaps due to a bug in tel2sip function.
-- juha
What version are you using?
It looks like a buffer overflow somewhere. Can you give the output of next commands in gdb:
frame 3 p *f
Cheers, Daniel
On 14/05/14 21:19, Juha Heinanen wrote:
i just noticed that my proxy had crashed on invite request from attacker:
May 14 22:03:06 sars /usr/sbin/sip-proxy[10932]: INFO: INVITE tel:004412127921\ 94 by untrusted sip:210.125.64.233 from <210.125.64.233> May 14 22:03:06 sars /usr/sbin/sip-proxy[10932]: : <core> [mem/q_malloc.c:159]:\ qm_debug_frag(): BUG: qm_*: prev. fragm. tail overwritten(c0c0c000, abcdefed)[\ 0xb70f6a64:0xb70f6a7c]! May 14 22:03:08 sars /usr/sbin/sip-proxy[11014]: : <core> [pass_fd.c:293]: rece\ ive_fd(): ERROR: receive_fd: EOF on 24 May 14 22:03:08 sars /usr/sbin/sip-proxy[10913]: ALERT: <core> [main.c:775]: ha\ ndle_sigs(): child process 10932 exited by a signal 6 May 14 22:03:08 sars /usr/sbin/sip-proxy[10913]: ALERT: <core> [main.c:778]: ha\ ndle_sigs(): core was generated
Program terminated with signal 6, Aborted. #0 0xb7782424 in __kernel_vsyscall () (gdb) where #0 0xb7782424 in __kernel_vsyscall () #1 0xb7616941 in raise () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #2 0xb7619d72 in abort () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 #3 0x08179f86 in qm_debug_frag (qm=0xb6dea008, f=0xb70f6a64) at mem/q_malloc.c:161 #4 0x0817ac3a in qm_malloc (qm=0xb6dea008, size=48, file=0x81f3169 "<core>: action.c", func=0x81f42f0 "do_action", line=780) at mem/q_malloc.c:386 #5 0x0805e798 in do_action (h=0xbffc1ca0, a=0xbffc1d48, msg=0xb72c6928) at action.c:780 #6 0xb1c5ac1d in pv_set_ruri (msg=0xb72c6928, param=0xb6f75630, op=254, val=0xbffc1e0c) at pv_core.c:2019 #7 0xb1b5df59 in tel2sip (_msg=0xb72c6928, _uri=0xb6f75c70 "H\207\367\266\004", _hostpart=0xb6f75530 "\254y\367\266\004", _res=0xb6f75624 "\006") at checks.c:405 #8 0x0805fdf7 in do_action (h=0xbffc21e0, a=0xb6f77858, msg=0xb72c6928) at action.c:1117 #9 0x08067293 in run_actions (h=0xbffc21e0, a=0xb6f77858, msg=0xb72c6928) at action.c:1599 #10 0x080678e2 in run_actions_safe (h=0xbffc39ac, a=0xb6f77858, msg=0xb72c6928) at action.c:1664 #11 0x081015fe in rval_get_int (h=0xbffc39ac, msg=0xb72c6928, i=0xbffc2528, rv=0xb6f779fc, cache=0x0) at rvalue.c:924 #12 0x08103f83 in rval_expr_eval_int (h=0xbffc39ac, msg=0xb72c6928, res=0xbffc2528, rve=0xb6f779f8) at rvalue.c:1918 #13 0x0810416e in rval_expr_eval_int (h=0xbffc39ac, msg=0xb72c6928, res=0xbffc27c4, rve=0xb6f78360) at rvalue.c:1926 #14 0x0805fa26 in do_action (h=0xbffc39ac, a=0xb6f78820, msg=0xb72c6928) at action.c:1075 #15 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f78820, msg=0xb72c6928) at action.c:1599 #16 0x0805fca0 in do_action (h=0xbffc39ac, a=0xb6f788c4, msg=0xb72c6928) at action.c:1094 ---Type <return> to continue, or q <return> to quit--- #17 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f788c4, msg=0xb72c6928) at action.c:1599 #18 0x0805fc5f in do_action (h=0xbffc39ac, a=0xb6f78968, msg=0xb72c6928) at action.c:1090 #19 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6f78968, msg=0xb72c6928) at action.c:1599 #20 0x0805e00d in do_action (h=0xbffc39ac, a=0xb6e7720c, msg=0xb72c6928) at action.c:715 #21 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e5161c, msg=0xb72c6928) at action.c:1599 #22 0x0805e00d in do_action (h=0xbffc39ac, a=0xb6e50238, msg=0xb72c6928) at action.c:715 #23 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e50238, msg=0xb72c6928) at action.c:1599 #24 0x0805fc5f in do_action (h=0xbffc39ac, a=0xb6e50bfc, msg=0xb72c6928) at action.c:1090 #25 0x08067293 in run_actions (h=0xbffc39ac, a=0xb6e4891c, msg=0xb72c6928) at action.c:1599 #26 0x0806797a in run_top_route (a=0xb6e4891c, msg=0xb72c6928, c=0x0) at action.c:1685 #27 0x080e2bcf in receive_msg ( buf=0x82f99e0 "INVITE tel:00441212792194 SIP/2.0\r\nVia: SIP/2.0/UDP 210.125.64.233;branch=z9hG4bK4KmbLm4c\r\nMax-Forwards: 69\r\nFrom: sip:210.125.64.233;tag=qua2A5c8s9VJZ\r\nTo: tel:00441212792194\r\nContact: <sip:210.1"..., len=1115, rcv_info=0xbffc3bb0) at receive.c:211 #28 0x081702cd in udp_rcv_loop () at udp_server.c:536 #29 0x080ad9a0 in main_loop () at main.c:1617 #30 0x080b098f in main (argc=17, argv=0xbffc3e64) at main.c:2533
perhaps due to a bug in tel2sip function.
-- juha
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Daniel-Constantin Mierla writes:
What version are you using?
this is version 4.1 taken from git on may 12.
It looks like a buffer overflow somewhere. Can you give the output of next commands in gdb:
frame 3 p *f
i got this:
(gdb) frame 3 #3 0x08179f86 in qm_debug_frag (qm=0xb6dea008, f=0xb70f6a64) at mem/q_malloc.c:161 161 mem/q_malloc.c: No such file or directory. (gdb) p *f $1 = {size = 60, u = {nxt_free = 0xb6dea32c, is_free = -1226923220}, file = 0xb1b6ea2f "siputils: checks.c", func = 0x823e2a3 "fragm. from qm_malloc", line = 383, check = 4042322160}
-- juha
The issue was with previous fragment (misread the log message in the first place). But was easy to spot what could be the previous fragment and I think I fixed with commit:
- http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=7992a2b8...
If you can give it a try, then it can be backported (I had no option to try it here for now).
Cheers, Daniel
On 15/05/14 06:51, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
What version are you using?
this is version 4.1 taken from git on may 12.
It looks like a buffer overflow somewhere. Can you give the output of next commands in gdb:
frame 3 p *f
i got this:
(gdb) frame 3 #3 0x08179f86 in qm_debug_frag (qm=0xb6dea008, f=0xb70f6a64) at mem/q_malloc.c:161 161 mem/q_malloc.c: No such file or directory. (gdb) p *f $1 = {size = 60, u = {nxt_free = 0xb6dea32c, is_free = -1226923220}, file = 0xb1b6ea2f "siputils: checks.c", func = 0x823e2a3 "fragm. from qm_malloc", line = 383, check = 4042322160}
-- juha
Daniel-Constantin Mierla writes:
The issue was with previous fragment (misread the log message in the first place). But was easy to spot what could be the previous fragment and I think I fixed with commit:
http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=7992a2b8...
If you can give it a try, then it can be backported (I had no option to try it here for now).
daniel,
thanks for spotting the bug. the problem with testing is that i not managed to reproduce it in master, but need to wait for the attacker to do the testing in my 4.1 setup.
the patch is very simple (allocate one more byte of space) and i cannot see how it would cause any problems. it is clear by reading the code that if no modifications are done, there is no space in the buffer for '\0'.
so i would suggest that the patch is cherry-picked to 4.1 now and i'll then keep watch on syslog for this attack in my 4.1 setup.
-- juha
On 15/05/14 14:14, Juha Heinanen wrote:
Daniel-Constantin Mierla writes:
The issue was with previous fragment (misread the log message in the first place). But was easy to spot what could be the previous fragment and I think I fixed with commit:
http://git.sip-router.org/cgi-bin/gitweb.cgi/sip-router/?a=commit;h=7992a2b8...
If you can give it a try, then it can be backported (I had no option to try it here for now).
daniel,
thanks for spotting the bug. the problem with testing is that i not managed to reproduce it in master, but need to wait for the attacker to do the testing in my 4.1 setup.
the patch is very simple (allocate one more byte of space) and i cannot see how it would cause any problems. it is clear by reading the code that if no modifications are done, there is no space in the buffer for '\0'.
so i would suggest that the patch is cherry-picked to 4.1 now and i'll then keep watch on syslog for this attack in my 4.1 setup.
Indeed, it is simple patch, but being in a hurry at that time I wanted to be sure there was no stupid mistake and cherry-pick some bug to stable branch. As you reviewed as well, I will backport shortly.
Cheers, Daniel
there was one more crash during the night caused by the same invite from attacker.
i reproduced the tel2sip call that my sip proxy runs, but i didn't get the crash:
$ru = "tel:00441212792194"; $avp(from_uri) = "sip:210.125.64.233"; xlog("L_INFO", "tel2sip $ru, $(avp(from_uri){uri.host}), $ru\n"); tel2sip("$ru", "$(avp(from_uri){uri.host})", "$ru");
May 15 08:56:23 siika /usr/sbin/sip-proxy[10685]: INFO: tel2sip tel:00441212792194, 210.125.64.233, tel:00441212792194
so i wonder why the crash happens on the attacker invite. perhaps it is not caused by tel2sip function by itself.
the line referred to in gdb where is the if line below:
/* set result pv value and write sip uri to result pv */ res_val.rs = sip_uri; res_val.flags = PV_VAL_STR; if (res->setf(_msg, &res->pvp, (int)EQ_T, &res_val) != 0) { LM_ERR("failed to set result pvar\n"); pkg_free(sip_uri.s); return -1; }
-- juha
-
Daniel-Constantin Mierla -
Juha Heinanen