On 2018-02-20 04:45 PM, Jeff Brower wrote:

It's great to see that rtpengine now supports transcoding. I always watched rtpproxy (for so many years) to see when it might happen.

I have some questions. I could not find on ffmpeg.org specific mention of:

-advanced audio jitter buffer capability

For the time being, rtpengine does not try to play nice on jitter. Audio data is decoded as soon as it arrives, passed to the encoder, and encoded data is sent out as soon as it's available. Depending on which codecs and packet sizes are involved, this may turn out transparent jitter-wise, or it may introduce considerable additional jitter. Dealing with jitter is thus left up to the endpoints.

-advanced RTP related RFCs, eg. RFC 8108

This would have to be handled in rtpengine itself and isn't directly related to transcoding.

-advanced codec capability, eg. G711 Appendix II (DTX and CNG)

This would also have to be handled in rtpengine as ffmpeg doesn't really support G.711 per se, but rather supports PCM a-Law und µ-Law as raw audio codecs. RFC 4733 will be handled in the near future.

In particular, it's not clear to me where jitter buffer and RTP RFCs are handled -- inside rtpgengine or ffmpeg. I don't see any mention of jitter buffer on the rtpengine web and github pages. In general ffmpeg's focus is on content delivery and not bidirectional real-time communication.

While I do believe that ffmpeg supports RTP in some way, this is not used by rtpengine -- ffmpeg is used only for raw codec support (and resampling).

1. Is there an rtpengine-ffmpeg software architecture or data flow diagram available ?

No, but you're welcome to submit one. :)

2. Is it possible to connect libraries besides ffmpeg to the "other side" of rtpengine ? For example, using the rtpengine interface, send and

receive a UDP/RTP packet stream to/from a third-party library and let it handle jitter buffer, encoding/decoding, ptime mismatch (transrating), and more ?

Well, it's just software, so anything is possible, right? :)

While this wasn't something we had on the radar when implementing this, I don't see why such a tie-in shouldn't be possible. At least non-ffmpeg codecs can be tied in relatively easily, which is what we did with bcg729.

3. If architecturally that's a do-able thing, is there a spec on how rtpengine is currently interfacing to ffmpeg ? Which APIs are being used ? (I

assume the command line interface is not being used). Re. ffmpeg APIs I found this:

http://dranger.com/ffmpeg/

but maybe there is something more recent or rtpengine source code we can look at.

The most relevant API is the avcodec API, described at https://ffmpeg.org/doxygen/trunk/avcodec_8h.html together with the related structures (AVFrame and AVPacket).

Also used is avresample/swresample from https://ffmpeg.org/doxygen/trunk/avresample_8h.html and the audio FIFO API https://ffmpeg.org/doxygen/trunk/audio__fifo_8h.html

The guts of how rtpengine ties into avcodec are https://github.com/sipwise/rtpengine/blob/master/lib/codeclib.c and most of the interfaces to the rest of rtpengine (SDP and control stuff) are at https://github.com/sipwise/rtpengine/blob/master/daemon/codec.c

HTH Cheers