首页 > 代码库 > 让 WebRTC 使用外部的音视频编解码器

让 WebRTC 使用外部的音视频编解码器

WebRTC 支持使用自己的编解码器(限 native 开发),音频,视频都可以。这里以视频编码为例来分析下 WebRTC 中相应的源码。

CreatePeerConnectionFactory

在 webrtc/api/peerconnectioninterface.h 中有个方法 CreatePeerConnectionFactory,原型如下:

inline rtc::scoped_refptr<PeerConnectionFactoryInterface> 
CreatePeerConnectionFactory(
    rtc::Thread* worker_and_network_thread,
    rtc::Thread* signaling_thread,
    AudioDeviceModule* default_adm,
    rtc::scoped_refptr<AudioEncoderFactory> audio_encoder_factory,
    rtc::scoped_refptr<AudioDecoderFactory> audio_decoder_factory,
    cricket::WebRtcVideoEncoderFactory* video_encoder_factory,
    cricket::WebRtcVideoDecoderFactory* video_decoder_factory);

可以看到, CreatePeerConnectionFactory 最后四个参数,允许我们提供自己的编解码工厂。这样我们就可以实现自己的 factory ,在 factory 中创建自己的 encoder(decoder),WebRTC 内部就会使用我们的 encoder(decoder)。

代码片段类似下面:

    rtc::scoped_refptr<webrtc::PeerConnectionFactoryInterface> factory = webrtc::CreatePeerConnectionFactory(
            current_thread, current_thread, 
            your_adm, 
            your_audio_encoder_factory, 
            your_audio_decoder_factory,
            your_video_encoder_factory,
            your_video_decoder_factory);
    rtc::scoped_refptr<PeerConnectionInterface> peer_connection = factory->CreatePeerConnection(
            config, constraints, NULL, NULL, your_observer);

WebRtcVideoEncoderFactory

这个接口在 webrtcvideoencoderfactory.h(webrtc/media/engine) 中定义:

class WebRtcVideoEncoderFactory {
 public:
  // This VideoCodec class is deprecated. Use cricket::VideoCodec directly
  // instead and the corresponding factory function. See
  // http://crbug/webrtc/6402 for more info.
  struct VideoCodec {
    webrtc::VideoCodecType type;
    std::string name;

    VideoCodec(webrtc::VideoCodecType t, const std::string& nm)
        : type(t), name(nm) {}

    VideoCodec(webrtc::VideoCodecType t,
               const std::string& nm,
               int w,
               int h,
               int fr)
        : type(t), name(nm) {}
  };

  virtual ~WebRtcVideoEncoderFactory() {}

  // TODO(magjed): Make these functions pure virtual when every external client
  // implements it. See http://crbug/webrtc/6402 for more info.
  // Caller takes the ownership of the returned object and it should be released
  // by calling DestroyVideoEncoder().
  virtual webrtc::VideoEncoder* CreateVideoEncoder(
      const cricket::VideoCodec& codec);

  // Returns a list of supported codecs in order of preference.
  virtual const std::vector<cricket::VideoCodec>& supported_codecs() const;

  // Caller takes the ownership of the returned object and it should be released
  // by calling DestroyVideoEncoder().
  // Deprecated: Use cricket::VideoCodec as argument instead. See
  // http://crbug/webrtc/6402 for more info.
  virtual webrtc::VideoEncoder* CreateVideoEncoder(webrtc::VideoCodecType type);

  // Returns a list of supported codecs in order of preference.
  // Deprecated: Return cricket::VideoCodecs instead. See
  // http://crbug/webrtc/6402 for more info.
  virtual const std::vector<VideoCodec>& codecs() const;

  // Returns true if encoders created by this factory of the given codec type
  // will use internal camera sources, meaning that they don‘t require/expect
  // frames to be delivered via webrtc::VideoEncoder::Encode. This flag is used
  // as the internal_source parameter to
  // webrtc::ViEExternalCodec::RegisterExternalSendCodec.
  virtual bool EncoderTypeHasInternalSource(webrtc::VideoCodecType type) const {
    return false;
  }

  virtual void DestroyVideoEncoder(webrtc::VideoEncoder* encoder) = 0;

 private:
  // TODO(magjed): Remove these. They are necessary in order to return a const
  // reference to a std::vector in the default implementations of codecs() and
  // supported_codecs(). See http://crbug/webrtc/6402 for more info.
  mutable std::vector<VideoCodec> encoder_codecs_;
  mutable std::vector<cricket::VideoCodec> codecs_;
};

实现 WebRtcVideoEncoderFactory 接口,传递给 CreatePeerConnectionFactory 即可。可以参考 webrtc/media/engine 目录下的 internalencoderfactory.h 和 internalencoderfactory.cpp 。

外部编解码器与内部的关系

以视频编码为例, webrtc/media/engine/webrtcvideoengine2.cc 中有段代码可以说明外部视频编码器和内部视频编码器之间的关系:

WebRtcVideoChannel2::WebRtcVideoSendStream::AllocatedEncoder
WebRtcVideoChannel2::WebRtcVideoSendStream::CreateVideoEncoder(
    const VideoCodec& codec) {
  RTC_DCHECK_RUN_ON(&thread_checker_);
  // Do not re-create encoders of the same type.
  if (codec == allocated_encoder_.codec &&
      allocated_encoder_.encoder != nullptr) {
    return allocated_encoder_;
  }

  // Try creating external encoder.
  if (external_encoder_factory_ != nullptr &&
      FindMatchingCodec(external_encoder_factory_->supported_codecs(), codec)) {
    webrtc::VideoEncoder* encoder =
        external_encoder_factory_->CreateVideoEncoder(codec);
    if (encoder != nullptr)
      return AllocatedEncoder(encoder, codec, true /* is_external */);
  }

  // Try creating internal encoder.
  if (FindMatchingCodec(internal_encoder_factory_->supported_codecs(), codec)) {
    if (parameters_.encoder_config.content_type ==
            webrtc::VideoEncoderConfig::ContentType::kScreen &&
        parameters_.conference_mode && UseSimulcastScreenshare()) {
      // TODO(sprang): Remove this adapter once libvpx supports simulcast with
      // same-resolution substreams.
      WebRtcSimulcastEncoderFactory adapter_factory(
          internal_encoder_factory_.get());
      return AllocatedEncoder(adapter_factory.CreateVideoEncoder(codec), codec,
                              false /* is_external */);
    }
    return AllocatedEncoder(
        internal_encoder_factory_->CreateVideoEncoder(codec), codec,
        false /* is_external */);
  }

  // This shouldn‘t happen, we should not be trying to create something we don‘t
  // support.
  RTC_NOTREACHED();
  return AllocatedEncoder(NULL, cricket::VideoCodec(), false);
}

可以看到,这里会先尝试调用 external_encoder_factory_ 创建外部的编码器,如果创建成功,用外部的,如果失败,再找内部的。

这里的 external_encoder_factory_ ,就是我们调用 CreatePeerConnectionFactory 时传入的那个 encoder factory 实例。


相关阅读:

  • WebRTC学习资料大全
  • Ubuntu 14.04下编译WebRTC
  • WebRTC源码中turnserver的使用方法
  • 打开 WebRTC 的日志(native api)
  • 让WebRTC支持H264编解码
  • WebRTC编译系统之gn和ninja
  • WebRTC编译系统之gn files
  • How to enable TRACE_EVENT in WebRTC codes
<script type="text/javascript"> $(function () { $(‘pre.prettyprint code‘).each(function () { var lines = $(this).text().split(‘\n‘).length; var $numbering = $(‘
    ‘).addClass(‘pre-numbering‘).hide(); $(this).addClass(‘has-numbering‘).parent().append($numbering); for (i = 1; i <= lines; i++) { $numbering.append($(‘
  • ‘).text(i)); }; $numbering.fadeIn(1700); }); }); </script>

    让 WebRTC 使用外部的音视频编解码器