首页 > 代码库 > webrtc音视频解析流程分析
webrtc音视频解析流程分析
webrtc音视频解析流程包括多个线程：
1. rtp网络流接收线程（rtp stream reciever thread）
2. 音视频解码线程（decode thread）
3. 渲染线程（render thread）
rtp网络流接收线程（rtp stream reciever thread）：
接收网络rtp包，解析rtp包，得到音视频数据包。将解析出的rtp包，加入到RtpStreamReceiver::frame_buffer_中或最终加入VCMReceiver::jitter_buffer_，解码线程从frame_buffer_或jitter_buffer_中取出帧进行解码。
涉及类：RtpStreamReceiver(rtp_stream_receiver.cc)， VideoReceiveStream（video_receive_stream.cc）
函数调用流程：
VideoReceiveStream::DeliverRtp() =>
rtp_stream_receiver_.DeliverRtp (rtp_stream_receiver.cc) =>
RtpStreamReceiver::ReceivePacket(rtp_stream_reciver.cc) =>
rtp_receiver_->IncomingRtpPacket(rtp_receiver_impl.cc) =>
rtp_media_receiver_->ParseRtpPacket(rtp_receiver_video.cc) =>
- depacketizer->Parse(rtp_format_h264.cc) 解析出payload_type, 如sps和pps等。
- data_callback_->OnReceivedPayloadData(rtp_stream_reciver.cc) =>
a) if(h264) InsertSpsPpsIntoTracker
packet_buffer_->InsertPacket
b) video_receiver_->IncomingPacket(video_receiver.cc) =>
_receiver.InsertPacket (receiver.cc) =>
上面a)和b)步骤是或关系，
a)步骤：
packet_buffer_->InsertPacket（packet_buffer.cc） =>
received_frame_callback_->OnReceivedFrame(rtp_stream_receiver.cc) =>
reference_finder->ManageFrame(rtp_frame_reference_finder.cc) =>
RtpFrameReferenceFinder::ManageFrame =>
a) RtpFrameReferenceFinder::ManageFrameGeneric =>
b) RtpFrameReferenceFinder::ManageFrameV8=>RtpFrameReferenceFinder::CompletedFrameV8 =>
c) RtpFrameReferenceFinder::ManageFrameV9=>RtpFrameReferenceFinder::CompletedFrameV9 =>
frame_callback_->OnCompleteFrame =>
complete_frame_callback_->OnCompleteFrame =>
frame_buffer_->InsertFrame
b)步骤：
_receiver.InsertPacket(receiver.cc) =>
jitter_buffer_.InsertPacket(jitter_buffer.cc) =>
decodable_frames_->InsertFrame(jitter_buffer.cc)
成员说明：
rtp_receiver_： RtpReceiver, 子类RtpReceiverImpl。
rtp_media_receiver_：RTPReceiverStrategy指针，子类是RTPReceiverAudio和RTPReceiverVideo。
data_callback： RtpData, 子类RtpStreamReceiver。
video_receiver_：VideoReceiver。
packet_buffer_: PacketBuffer。
_receiver： VCMReceiver。
received_frame_callback_: 类RtpStreamReceiver，实现video_coding::OnReceivedFrameCallback。
reference_finder_: 类RtpFrameReferenceFinder。
frame_callback_: 类RtpStreamReceiver，实现video_coding::OnCompleteFrameCallback。
complete_frame_callback_：类VideoReceiveStream， 实现video_coding::OnCompleteFrameCallback。
视频解码线程（video decode thread）：
从RtpStreamReceiver::frame_buffer_中读取每一帧进行解码。
涉及类：VideoReceiveStream(video_receive_stream.cc)
函数调用流程：
VideoReceiveStream::Decode(video_receive_stream.cc) =>
video_receiver_->Decode(video_receiver.cc) =>
1. _codecDataBase.GetDecoder(frame, _decodedFrameCallback) =>
ptr_decoder_->RegisterDecodeCompleteCallback(_decodedFrameCallback)
2. if (frame_buffer_->NextFrame) video_receiver_.Decode(frame) (从frame_buffer_取帧)
else video_receiver_.Decode(kMaxDecodeWaitTimeMs) (从jitter_buffer_取帧) =>
_receiver.FrameForDecoding(取帧) =>
jitter_buffer_.NextCompleteFrame
_deocoder->Decode() (_decoder是具体的decoder，如h264)=>
decoded_image_callback_->Decoded(generic_decoder.cc) =>
_receiveCallback->FrameToRender(video_stream_decoder.cc) =>
incoming_video_stream_->OnFrame(incoming_video_stream.cc) =>
render_buffers_->AddFrame(incoming_video_stream.cc)
rtp_stream_receiver_->FrameDecoded(统计)
成员说明：
video_receiver_：VideoReceiver。
_decodedFrameCallback: 类VCMDecodedFrameCallback。
_receiveCallback: 类VCMReceiveCallback，子类VideoStreamDecoder。
incoming_video_stream_: rtc::VideoSinkInterface<VideoFrame>*,子类IncomingVideoStream, WebRtcVideoReceiveStream。
渲染线程（render thread）：
渲染线程从IncomingVideoStream::render_buffers_ 中读取帧，发送出去。
涉及类：IncomingVideoStream
IncomingVideoStream::IncomingVideoStreamProcess(incoming_video_stream.cc) =>
1. render_buffers_->FrameToRender() =>
2. external_callback_->OnFrame (在VideoReceiveStream::Start 中设置为VideoReceiveStream) =>
config_.renderer->OnFrame (video_receive_stream.cc, WebRtcVideoReceiveStream构造函数中设置config_.render 为自己, 最终传递给VideoReceiveStream::config_)=>
sink_->OnFrame
sink设置：
PeerConnection::CreateVideoReceiver(peerconnection.cc) =>
VideoRtpReceiver::VideoRtpReceiver(rtpreceiver.cc) =>
channel_->SetSink(broadcaster_) =>
WebRtcVideoChannel2::SetSink(ssrc, sink) =>
receive_streams_[ssrc]->SetSink(sink)(最终设置WebRtcVideoReceiveStream::sink_)
成员说明：
render_buffers_：std::list<VideoRenderFrames>
external_callback_: rtc::VideoSinkInterface<VideoFrame>*。
sink_: rtc::VideoSinkInterface<webrtc::VideoFrame>*, 通过WebRtcVideoReceiveStream::SetSink设置。
broadcaster_: rtc::VideoBroadcaster，继承VideoSinkInterface<webrtc::VideoFrame>，broadcast video frames to sinks 。
receive_streams_: std::map<uint32_t, WebRtcVideoReceiveStream*>。
webrtc音视频解析流程分析