Earlier this month, Apple joined the Alliance for Open Media (AOM), which is working on the next-generation AV1 video compression standard. It’s interesting that Apple is now supporting AV1, after having just announced backing HEVC in both devices and software tools (i.e., publishing, browsers).
Why Apple Now Stands Behind AV1
There are a few reasons, I think, Apple chose to support AV1. Clearly, Apple has limitations when it comes to UHD on YouTube. YouTube only encodes UHD in VP9, a compression format that no Apple device or browser is capable of decoding. Moreover, Netflix has clearly stated that as soon as AV1 is available, it will initially support AV1 in browsers and then on connected devices once hardware support is available. Netflix is currently paying HEVC royalties, and if it can reduce these costs by using a different codec, without impacting the QoE, that’s a win-win situation. Apple knows it needs to be ready for AV1, especially if it wants to support Netflix services in the future, compete with Netflix, or even acquire the video streaming giant, as speculated about in the press recently.
If Apple did want to have its own OTT service, and the devices only support AV1 and not HEVC, they will cut themselves from those devices. That’s not a great way to enter the OTT market.
AV1 Mythbusters
Now let us break some of the AV1 myths. First, the performance mentioned by AOM sources as being 25 to 35 percent better than HEVC has to be verified by independent sources. At IBC2017, B<>COM demonstrated a side-by-side comparison between AV1 and HEVC on critical sequences of sports content and highly detailed static scenes. AV1 was clearly behind in performance at the same bitrate, or at the same Video Multimethod Assessment Fusion (VMAF) at a higher bitrate. AV1 might have hit a bump in the road here.
One important aspect when it comes to adopting a codec is the decoding device support. All of the popular OTT devices (i.e., smartphones, tablets, IP STBs and TVs) have a hardware decoder. The AV1 decoder is much more complex than HEVC and will require new devices. Those chips have to be built and integrated into devices that need to be manufactured. This will take at least two years, so AV1 hardware decoders will not see mass volume before 2020. AV1 has hit a second bump in the road here, because the universality of a codec is key. If the AV1 decoder BoM (decoding system plus royalties) is higher than the HEVC BoM, then it will be a non-starter. If the AV1 decoder system cost (i.e., chip plus memory) is too high, you will only see AV1 on high-end devices (e.g., UHD TVs and high-powered smartphones). It won’t exist on the lower end, where AVC might reign for another cycle, especially at HD resolution, using content-aware encoding techniques.
Encoding costs are also important to consider. Today, the AV1 reference model is 100 times slower than an optimized HEVC encoder. We expect the additional complexity of the encoder to be around 10 times vs HEVC. This might work for VOD in the cloud where more encoding resources are available, but will not fly for live on-premise applications. AV1 has hit another bump here, because if the encoding cost is too high, broadcasters and service providers will stay in the AVC/HEVC world. Side note: AV1 is only for ABR and does not support broadcast.
Finally, let’s take a look at the legal aspect. AV1 has taken extra precautions to make sure no patents are infringed. If you look at the legal construction of the AOM, it uses an umbrella patent scheme, meaning the AOM licensors license their AV1-connected patents to anyone, anywhere, anytime based on reciprocity, i.e. as long as the user does not engage in patent litigation. As a defensive condition, anyone engaging in patent litigation loses the right to the patents of all patent holders. With Apple now onboard with the AOM, free access will be given to its MPEG patent, which could be crucial for the AOM’s success. If you add to that the fact that Technicolor (another large MPEG patent holder) is selling its licensing division, the AOM (or one of its members) could acquire those valuable patents and make the AOM patent portfolio even stronger.
What Does All This Mean?
Is the AV1 announcement by Apple as big of a deal as its support of HEVC? I don’t think so for several reasons: We do not know Apple’s intentions, AV1 faces serious roadblocks and AV1’s impact will not be meaningful before 2020. Until then, there is room for enhancement of AVC.
However, one thing is clear: If HEVC patent licensors thought they would have a nice ride to the bank, they now know that is not going to be the case. Furthermore, for the last year, firms like Unified Patents are providing the industry with useful programs and resources that may help rationalize the HEVC patent landscape. As an industry, we hope that the HEVC licensors come up with a reasonable and fair solution to resolving the patent licensing issues.
Who said video codecs were boring? The video codec ecosystem is just heating up!
Thierry Fautier, Vice President of Video Strategy at Harmonic and President of the Ultra HD Forum