Hello,
I am excited about the new media statistics from RTPEngine, exposed conveniently into PVs, e.g. jitter, packet loss, MOS, time offset of lowest and highest MOS scores, etc. Great work!
I am, however, having a bit of trouble understanding how to properly apply this new functionality, as it is not clearly documented (IMHO):
1. What is the meaning of "global" statistics which do not take into account an "A" and a "B" side? That is to say, if I have a packet loss or a jitter or MOS score statistic using just the generic toplevel values, e.g.
https://kamailio.org/docs/modules/5.1.x/modules/rtpengine.html#rtpengine.p.m...
What does that mean?
A functional call ordinarily consists of an inbound and outbound RTP stream direction relative to the endpoints on both sides of RTPEngine.
Inbound A --> server --> Outbound A Outbound B <-- server <--- Inbound B
Where do the "global" stats situate themselves in relation to all this?
2. I am given to understand that the statistics for the "A" and "B" sides of a bidirectional RTP stream can be separated based on an arbitrary label, using these PVs:
https://kamailio.org/docs/modules/5.1.x/modules/rtpengine.html#rtpengine.p.m...
https://kamailio.org/docs/modules/5.1.x/modules/rtpengine.html#rtpengine.p.m...
How do I use it properly?
My intuition was to try something like this:
rtpengine_offer("replace-origin replace-session-connection ICE=remove label=outbound");
And in the answer/manage handler on the SDP answer:
rtpengine_answer("replace-origin replace-session-connection ICE=remove label=inbound");
Later, when processing a BYE and obtaining subsequent media statistics, I tried this:
if(check_route_param("proxy_media=yes")) { $avp(rtpengine_a_label) = 'outbound'; $avp(rtpengine_b_label) = 'inbound'; rtpengine_delete(); route(LOG_RTPENGINE_STATS); }
But this only gets me stats for the "A" leg in the JSON object I emit:
{ "server_addr": "udp:127.0.0.1:5050", "a_label": "outbound", "b_label": "inbound", "a_leg": { "min": { "mos": "4.4", "at": "0:04", "packetloss": "0", "jitter": "0", "roundtrip": 0 }, "max": { "mos": "4.4", "at": "0:04", "packetloss": "0", "jitter": "0", "roundtrip": 0 }, "avg": { "mos": "4.4", "packetloss": "0", "jitter": "0", "roundtrip": 0, "samples": 2 } }, "b_leg": { "min": { "mos": "<null>", "at": "<null>", "packetloss": "<null>", "jitter": "<null>" }, "max": { "mos": "<null>", "at": "<null>", "packetloss": "<null>", "jitter": "<null>" }, "avg": { "mos": "<null>", "packetloss": "<null>", "jitter": "<null>" } } }
I also tried rtpengine_delete("label=inbound") (and outbound) individually.
Some clarity on how this all fits together would be appreciated. :-) Many thanks to Sipwise for great work on it!
-- Alex
On 2018-02-13 06:16 PM, Alex Balashov wrote: ...
It's the min, max, and average between the two call legs (and other contributing sources that may have been involved in the call).
That should actually work. You can check the actual data returned from rtpengine to Kamailio by running rtpengine with log level 7, or use the included rtpengine-ng-client script to run a "query" command against rtpengine for a running call and see what comes out of it.
Cheers
Hi Richard,
On Tue, Feb 13, 2018 at 06:34:53PM -0500, Richard Fuchs wrote:
It's the min, max, and average between the two call legs (and other contributing sources that may have been involved in the call).
When you say "between", do you mean the average of the values of the two respective legs?
Thank you, I will investigate further.
I had a preliminary look at the netstring just by sniffing the control protocol traffic, and saw that the stats were indeed being separated into the respective "inbound" and "outbound" classifications. But for some reason that was not percolating back into Kamailio PVs.
-- Alex
On Tue, Feb 13, 2018 at 07:34:30PM -0500, Richard Fuchs wrote:
The average is the average between the two, the min is the min between the two, and the max is the max between the two.
Ach so! Jetzt alles ist klar!
After I finally got the Perl client to work, unfortunately it raised more questions than answers. :-)
[root@evaristesys-legacy rtpengine]# perl utils/rtpengine-ng-client list Result dictionary: -----8<-----8<-----8<-----8<-----8<----- { result => 'ok', calls => [ '835853f6cbf2a03ac8329bb92b0d1c8a' ] } ----->8----->8----->8----->8----->8-----
[root@evaristesys-legacy rtpengine]# perl utils/rtpengine-ng-client query --call-id=835853f6cbf2a03ac8329bb92b0d1c8a Error reason: "Unknown call-id" at /opt/rh/rh-perl524/root/usr/share/perl5/NGCP/Rtpengine.pm line 43.
Feb 14 02:28:38 evaristesys-legacy rtpengine[29025]: WARNING: [SCALAR(0x855cd8)]: Protocol error in packet from 127.0.0.1:58000: Unknown call-id [d7:call-id16:SCALAR(0x855cd8)7:command5:querye]
-- Alex
On Tue, Feb 13, 2018 at 07:34:30PM -0500, Richard Fuchs wrote:
Ah, okay, coercing it to use Convert::Bencode (for lack of CentOS rh524-perl Bencode module) was not a productive thing to do. :-)
On Tue, Feb 13, 2018 at 09:29:49PM -0500, Alex Balashov wrote:
Well, both the 'query' command and final logging confirm that the labels are correct for the respective from-tags:
Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: Final packet stats: Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --- Tag 'F00E08C8-2966E6E7' (label 'outbound'), created 0:34 ago for branch '', in dialogue with 'sansay1987955648rdb39660' Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: ------ Media #1 (audio over RTP/AVP) using PCMU/8000 Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --------- Port 4.5.6.7:8118 <> 47.39.154.156:2234 , SSRC 4149441922, 112 p, 19264 b, 0 e, 30 ts Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --------- Port 4.5.6.7:8119 <> 47.39.154.156:2235 (RTCP), SSRC 4149441922, 1 p, 84 b, 0 e, 30 ts Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --- Tag 'sansay1987955648rdb39660' (label 'inbound'), created 0:34 ago for branch '', in dialogue with 'F00E08C8-2966E6E7' Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: ------ Media #1 (audio over RTP/AVP) using PCMU/8000 Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --------- Port 4.5.6.7:8098 <> 8.9.10.11:63736, SSRC 790287681, 438 p, 75336 b, 0 e, 24 ts Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --------- Port 4.5.6.7:8099 <> 8.9.10.11:63737 (RTCP), SSRC 0, 0 p, 0 b, 0 e, 34 ts Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: --- SSRC 4149441922 Feb 14 05:10:32 evaristesys-legacy rtpengine[7354]: INFO: [c6fa30199831602c397d627d5a2662d3]: ------ Average MOS 4.4, lowest MOS 4.4 (at 0:03), highest MOS 4.4 (at 0:03)
But for some reason, the "B" PVs just aren't being populated, even though I use the labels:
if(check_route_param("proxy_media=yes")) { $avp(rtpengine_a_label) = 'outbound'; $avp(rtpengine_b_label) = 'inbound';
rtpengine_delete();
route(LOG_RTPENGINE_STATS); }
And have the PVs referenced:
modparam("rtpengine", "mos_A_label_pv", "$avp(rtpengine_a_label)") modparam("rtpengine", "mos_B_label_pv", "$avp(rtpengine_b_label)")
This is using rtpengine:mr5.5.3 and kamailio:5.1 latest clone.
-- Alex
On Tue, Feb 13, 2018 at 09:39:59PM -0500, Alex Balashov wrote:
All right, I think what I have learned is that the stats simply aren't being collected for the SSRC in the endpoint that corresponds to my "inb" (inbound) label — just the "outb" (outbound):
Here's the complete dump. Note in the MOS stats that the stats for this SSRC are empty:
222410830 => {}
I did note that there are two SSRCs for the more viable outbound stream, one being RTP and the other being RTCP. The other direction also has an RTP and an RTCP stream, but an SSRC only for one of them.
See below:
-----8<-----8<-----8<-----8<-----8<----- { created => '1518590122', 'last signal' => '1518590124', tags => { sansay1987983383rdb28749 => { label => 'inb', medias => [ { type => 'audio', protocol => 'RTP/AVP', index => '1', flags => [ 'initialized', 'send', 'recv' ], streams => [ { stats => { packets => '1581', bytes => '271932', errors => '0' }, endpoint => { address => '<ITSP>', family => 'IPv4', port => '38930' }, 'advertised endpoint' => { address => '<ITSP>', port => '38930', family => 'IPv4' }, 'last packet' => '1518590156', 'local port' => '9078', flags => [ 'RTP', 'filled', 'confirmed', 'kernelized' ], SSRC => '222410830' }, { stats => { errors => '0', bytes => '0', packets => '0' }, 'advertised endpoint' => { family => 'IPv4', port => '38931', address => '<ITSP>' }, endpoint => { family => 'IPv4', port => '38931', address => '<ITSP>' }, 'last packet' => '1518590122', 'local port' => '9079', flags => [ 'RTCP', 'filled' ] } ] } ], created => '1518590122', tag => 'sansay1987983383rdb28749', 'in dialogue with' => '9C62E1A4-89AFCE63' }, '9C62E1A4-89AFCE63' => { 'in dialogue with' => 'sansay1987983383rdb28749', tag => '9C62E1A4-89AFCE63', created => '1518590122', medias => [ { protocol => 'RTP/AVP', type => 'audio', streams => [ { SSRC => '4088772468', flags => [ 'RTP', 'filled', 'confirmed', 'kernelized' ], 'local port' => '9088', 'last packet' => '1518590156', endpoint => { family => 'IPv4', port => '2250', address => '<home IP phone>' }, 'advertised endpoint' => { address => '172.30.105.251', family => 'IPv4', port => '2250' }, stats => { errors => '0', bytes => '270212', packets => '1571' } }, { SSRC => '4088772468', 'last packet' => '1518590152', 'local port' => '9089', flags => [ 'RTCP', 'filled', 'confirmed' ], endpoint => { family => 'IPv4', port => '2251', address => '<home IP phone>' }, 'advertised endpoint' => { port => '2251', family => 'IPv4', address => '172.30.105.251' }, stats => { errors => '0', bytes => '504', packets => '6' } } ], index => '1', flags => [ 'initialized', 'send', 'recv' ] } ], label => 'outb' } }, created_us => '624757', result => 'ok', SSRC => { '4088772468' => { 'highest MOS' => { 'packet loss' => '0', 'round-trip time' => '0', jitter => '2', 'reported at' => '1518590127', MOS => '44' }, 'lowest MOS' => { 'packet loss' => '0', 'round-trip time' => '0', 'reported at' => '1518590127', jitter => '2', MOS => '44' }, 'average MOS' => { MOS => '44', jitter => '2', samples => '6', 'round-trip time' => '0', 'packet loss' => '0' }, 'MOS progression' => { interval => '2', entries => [ { 'round-trip time' => '0', 'packet loss' => '0', 'reported at' => '1518590127', MOS => '44', jitter => '2' }, { 'packet loss' => '0', 'round-trip time' => '0', 'reported at' => '1518590132', MOS => '44', jitter => '3' }, { jitter => '0', 'reported at' => '1518590137', MOS => '44', 'round-trip time' => '0', 'packet loss' => '0' }, { 'reported at' => '1518590142', MOS => '44', jitter => '2', 'packet loss' => '0', 'round-trip time' => '0' }, { 'round-trip time' => '0', 'packet loss' => '0', 'reported at' => '1518590147', MOS => '44', jitter => '4' }, { 'round-trip time' => '0', 'packet loss' => '0', 'reported at' => '1518590152', jitter => '3', MOS => '44' } ] } }, 222410830 => {} }, totals => { RTCP => { errors => '0', bytes => '504', packets => '6' }, RTP => { errors => '0', bytes => '542144', packets => '3152' } } } ----->8----->8----->8----->8----->8-----
-- Alex
On Wed, Feb 14, 2018 at 12:15:29AM -0500, Alex Balashov wrote:
Okay, so after having a spelunk through the source code, I have the impression that this MOS / quality analysis functionality is limited solely to feedback off RTCP supports.
I was under the impression from the documentation that RTPEngine does some stream analysis of its own, although in fairness, the documentation does not explicitly make this claim or the contrary one way or another. :-)
But that would certainly explain my problem. It seems I fundamentally misunderstood what it is that RTPEngine actually provides.
On 2018-02-14 02:44 AM, Alex Balashov wrote:
It does gather some stats about the media streams on its own (even more so with the extended transcoding capabilities now), but at least one component of the MOS calculation (latency) depends on RTCP being provided by the RTP clients, so without that, no MOS score can be produced.
Cheers
Hi Richard,
On Wed, Feb 14, 2018 at 08:18:11AM -0500, Richard Fuchs wrote:
Which statistics can RTPEngine compute over the stream on its own? Since I saw no metrics whatsoever for the B leg, I assumed none. I was running mr5.5.3 however. Does this differ in 6+?
Also — I'm sure the question has been asked before — can RTPEngine send its own RTCP sender/receiver reports?
My testing setup was:
Phone --> Kamailio+RTPEngine --> Kamailio+RTPEngine --> Carrier (where tinkering (ITSP was happening)
Is there a way to turn on backward RTCP feedback from #3 back to #2?
On 2018-02-14 12:21 PM, Alex Balashov wrote:
It keeps track of packet and octet counts. With transcoding engaged for a call, it also keeps track of packet loss and duplications. It could calculate jitter, but doesn't currently do so.
Not currently, but we may offer that capability in the future, especially to support video in an RTP/AVP to (S)AVFP bridging scenario.
Cheers
On Wed, Feb 14, 2018 at 12:49:04PM -0500, Richard Fuchs wrote:
It keeps track of packet and octet counts.
Is that the stuff exposed via the traditional $rtpstat facility?
What I was really looking to ascertain is that none of the statistics available under the new rtpengine module PVs are populated without RTCP.
Aha, thank you!
Is this an aspect of the transcoding functionality, or simply non-kernelised relay?
-- Alex
On 2018-02-14 12:51 PM, Alex Balashov wrote:
Correct, these are all currently extracted from RTCP only.
Both are needed. Without transcoding engaged for a call, rtpengine ignores most aspects of RTP, including sequence numbers and timestamps, which would be needed for stats keeping.
Cheerse
-
Alex Balashov -
Richard Fuchs