Add frame.wsc == NULL check in ws_frame_transmit similar to the checks in ws_close and friends and ws_frame_receive to avoid crashing if the tcp connection was been terminated prematurely.
In the wild, the following was observed:
Jan 11 16:26:20 ws0 /usr/sbin/kamailio[16621]: ERROR: <core> [tcp_main.c:715]: _wbufq_add(): ERROR: wbufq_add(1077 bytes): write queue full or timeout (32122, total 32122, last write 0 s ago) Jan 11 16:26:20 ws0 /usr/sbin/kamailio[16621]: ERROR: websocket [ws_frame.c:299]: encode_and_send_ws_frame(): sending WebSocket frame Jan 11 16:26:20 ws0 /usr/sbin/kamailio[16621]: ERROR: websocket [ws_frame.c:740]: ws_frame_transmit(): sending message .... Jan 11 16:26:20 ws0 /usr/sbin/kamailio[16637]: ERROR: <core> [tcp_main.c:3638]: handle_ser_child(): handle_ser_child: ERROR: received CON_ERROR for 0x7fc500606808 (id 3492), refcnt 3, flags 0x601c Jan 11 16:26:20 ws0 /usr/sbin/kamailio[16628]: WARNING: <core> [tcp_read.c:1604]: handle_io(): WARNING: tcp_receive: handle_io: F_TCPCONN connection marked as bad: 0x7fc500606808 id 3492 refcnt 1
And then, 5 seconds later:
Jan 11 16:26:25 ws0 /usr/sbin/kamailio[16612]: ALERT: <core> [main.c:777]: handle_sigs(): child process 16618 exited by a signal 11
Backtrace:
#0 0x00007f8694ee4bfe in encode_and_send_ws_frame (frame=frame@entry=0x7fff1a295130, conn_close=conn_close@entry=CONN_CLOSE_DONT) at ws_frame.c:148 #1 0x00007f8694ee9598 in ws_frame_transmit (data=<optimized out>) at ws_frame.c:738 #2 0x00007f8695fc3e20 in msg_send (len=<optimized out>, buf=<optimized out>, dst=<optimized out>) at ../../forward.h:187 #3 send_pr_buffer (rb=rb@entry=0x7f86888730b0, buf=0x7f8688883388, len=<optimized out>) at t_funcs.c:102 #4 0x00007f8695fcf104 in t_send_branch (t=t@entry=0x7f8688872f38, branch=branch@entry=0, p_msg=p_msg@entry=0x7f86976212b0, proxy=proxy@entry=0x0, lock_replies=lock_replies@entry=1) at t_fwd.c:1580 #5 0x00007f8695fd2814 in t_forward_nonack (t=0x7f8688872f38, p_msg=p_msg@entry=0x7f86976212b0, proxy=proxy@entry=0x0, proto=proto@entry=0) at t_fwd.c:1790 #6 0x00007f8695fc50fa in t_relay_to (p_msg=0x7f86976212b0, proxy=0x0, proto=0, replicate=0) at t_funcs.c:354 #7 0x0000000000421980 in do_action (h=h@entry=0x7fff1a295c70, a=a@entry=0x7f86975e3798, msg=msg@entry=0x7f86976212b0) at action.c:1105
The same thing happened three times over the weekend. All with kamailio-4.1.6 from http://deb.kamailio.org/kamailio .
Could it be that this missing check was simply an oversight? Or is there a reason why the check wasn't done in ws_frame_transmit?
----
This patch was compile tested only. We haven't tried to reproduce the crash to see if this fixes it. But I figure this should at least stop the crash, and probably relay back that the destination doesn't exist anymore. You can merge this Pull Request by running:
git pull https://github.com/wdoekes/kamailio wjd-ws_frame_transmit-check-wsc
Or you can view, comment on it, or merge it online at:
https://github.com/kamailio/kamailio/pull/26
-- Commit Summary --
* websocket: Check frame->wsc in ws_frame_transmit for NULL.
-- File Changes --
M modules/websocket/ws_frame.c (5)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/26.patch https://github.com/kamailio/kamailio/pull/26.diff
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/26
Merged #26.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/26#event-218635840
Thanks for the merge.
Status update: since the patch, this has triggered once (a bunch of times in the same hour, actually). Kamailio survived and is still running. I haven't found the exact cause yet, I'll let you know if I do.
Question: I see you didn't cherry-pick this into 4.1/4.2. Do you need something from me to get this included in the live branches?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/26#issuecomment-70056915
Backported now (to 4.2 and 4.2) - I didn't have the time at that moment.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/26#issuecomment-70150324