21 Feb
2018
21 Feb
'18
12:18 a.m.
On 2018-02-20 04:45 PM, Jeff Brower wrote:
It's great to see that rtpengine now supports
transcoding. I always watched rtpproxy (for so many years) to see when it might happen.
I have some questions. I could not find on ffmpeg.org specific mention of:
-advanced audio jitter buffer capability
For the time being, rtpengine does not try to play nice on jitter. Audio
data is decoded as soon as it arrives, passed to the encoder, and
encoded data is sent out as soon as it's available. Depending on which
codecs and packet sizes are involved, this may turn out transparent
jitter-wise, or it may introduce considerable additional jitter. Dealing
with jitter is thus left up to the endpoints.
-advanced RTP related RFCs, eg. RFC 8108
This would have to be handled in rtpengine itself and isn't directly
related to transcoding.
-advanced codec capability, eg. G711 Appendix II
(DTX and CNG)
This would also have to be handled in rtpengine as ffmpeg doesn't really
support G.711 per se, but rather supports PCM a-Law und ยต-Law as raw
audio codecs. RFC 4733 will be handled in the near future.
In particular, it's not clear to me where jitter buffer and RTP RFCs are handled --
inside rtpgengine or ffmpeg. I don't see any mention of jitter
buffer on the rtpengine web and github pages. In general ffmpeg's focus is on
content delivery and not bidirectional real-time communication.
While I do believe that ffmpeg supports RTP in some way, this is not
used by rtpengine -- ffmpeg is used only for raw codec support (and
resampling).
1) Is there an rtpengine-ffmpeg software architecture or data flow diagram available ?
No, but you're welcome to submit one. :)
2) Is it possible to connect libraries besides ffmpeg to the "other side" of
rtpengine ? For example, using the rtpengine interface, send and
receive a UDP/RTP packet stream to/from a third-party library and let it handle jitter
buffer, encoding/decoding, ptime mismatch (transrating), and
more ?
Well, it's just software, so anything is possible, right? :)
While this wasn't something we had on the radar when implementing this,
I don't see why such a tie-in shouldn't be possible. At least non-ffmpeg
codecs can be tied in relatively easily, which is what we did with bcg729.
3) If architecturally that's a do-able thing, is there a spec on how rtpengine is
currently interfacing to ffmpeg ? Which APIs are being used ? (I
assume the command line interface is not being used). Re. ffmpeg APIs I found this:
http://dranger.com/ffmpeg/
but maybe there is something more recent or rtpengine source code we can look at.
The most relevant API is the avcodec API, described at
https://ffmpeg.org/doxygen/trunk/avcodec_8h.html together with the
related structures (AVFrame and AVPacket).
Also used is avresample/swresample from
https://ffmpeg.org/doxygen/trunk/avresample_8h.html and the audio FIFO
API https://ffmpeg.org/doxygen/trunk/audio__fifo_8h.html
The guts of how rtpengine ties into avcodec are
https://github.com/sipwise/rtpengine/blob/master/lib/codeclib.c and most
of the interfaces to the rest of rtpengine (SDP and control stuff) are
at https://github.com/sipwise/rtpengine/blob/master/daemon/codec.c
HTH
Cheers