This post is a part of a series on the Chromium rendering engine. The team is responsible for the web facing APIs for video playback like MSE and WebCodecs, and the platform specific internals involved in demuxing, decoding, and rendering audio and video. By Dale Curtis.
In this article, I’ll walk you through Chromium’s video rendering architecture. While some details around extensibility are likely Chromium-specific, most of the concepts and designs discussed here apply to other rendering engines and even native playback apps.
In the beginning, video rendering was quite simple - just a for loop choosing which software decoded video frames to send to the compositor. For years this was reliable enough, but as the complexity of the web increased, the need for more performance and efficiency led to architectural changes. Many improvements required OS-specific primitives; thus, our architecture also had to become more extensible to reach all of Chromium’s platforms.
![Chromium architecture evolution]https://wd.imgix.net/image/ZDZVuXt6QqfXtxkpXcPGfnygYjd2/IMFPhRwU3MKT1lEvfzIZ.png?auto=format&w=845 “Diagram of rendering flow to different Chromium platforms, Source: https://developer.chrome.com/blog/videong/")
Source: https://developer.chrome.com/blog/videong/
Chromium’s playback architecture has changed significantly over the years. While we didn’t start with the idea of a pyramid of success as described in the first post in this series, we ultimately followed similar steps: reliability, performance, and then extensibility.
We’ve focused on how Chromium takes advantage of OS primitives to deliver best in class playback experience. Very interesting!
[Read More]