H.264. HEVC. VP9. If these terms mean nothing to you, you’re not alone. Each is a kind of video codec. Though they’re responsible for every second of video we watch online or on a Blu-ray, most people sitting at home watching Netflix on their Rokus or smart TVs have little reason to think about them.
However, recent developments in the codec world are going to bring some exciting changes to way streaming video works, along with some potential pitfalls. Here’s everything you need to know.
What is a codec?
Before we start, let’s quickly discuss what codecs are, and why there seem to be so many different ways of labeling them.
If you already know a lot about this stuff, you may be jumping up and down, screaming, “Stop confusing codecs with video coding formats, they’re not the same!” You’re right, they’re different. But for most people, that distinction doesn’t mean much, so we’re going to stick with the term codec, with our apologies to the technical community.
The word codec is a portmanteau — a word that combines two other words — of “encode” and “decode.” It describes a method for taking a specific type of file (in this case a video file) and changing it through an encoding process. Encoding can accomplish a lot of things, but usually, it makes the file smaller by using compression techniques.
Of course, once a file has been encoded and sent over the internet or stored on a disc, it must also be decoded by whichever device or app is responsible for playing the video, and a codec describes the method for doing that part too.
If you’re familiar with the MP3 format then you’re already codec-savvy even if you didn’t know it. The MP3 audio format that took over the music world more than 20 years ago is actually an audio codec that can shrink a CD audio track down to less than a tenth of its original size.
What’s with all of those letters?
As technical standards, codecs tend to be given the kind of labels only an engineer could love. H.263, H.264, and H.265 are all examples of these labels. But along with their formal designations, codecs are often given a friendly name like “High-Efficiency Video Coding,” which naturally gets shortened to HEVC (hey, it’s all about making things smaller, right?).
The most famous example of a technical standard being given a friendly name is IEEE 802.11. For most of us, we simply know it as “Wi-Fi.”
Why should I care about codecs?
Frankly, the only people who should really care about codecs are the engineers who develop them and the hardware, software, and streaming service companies that need to implement them.
But codecs can have a direct impact on the quality of video we see, the amount of data that video uses up on our mobile or home data plans, and on our choices when we go to buy a new TV or streaming media device.
As such, it’s worth keeping an eye on developments in the video codec world to make sure you’re able to take advantage of the latest benefits that they offer.
OK, so what’s changing?
You can think of the evolution of codecs in both the audio and video world as a never-ending quest to find more efficient ways to reduce the size of a given file while preserving its quality.
The last major achievement in this quest happened in 2013 when H.265 — better known as HEVC — debuted. It boasted the ability to take a video file and compress it down to half of the size that the previous best codec (H.264 AVC) could achieve, and it did so without any loss in quality.
In real terms, this meant that if you streamed a 4K resolution video using H.264, it would require 32Mbps of bandwidth. Using HEVC, that same video only needed about 15Mbps.
But HEVC is now seven years old (which means the technology itself is older still) and it’s time for it to pass the torch to a new generation of codecs.
H.266 VVC
H.266 Versatile Video Coding (VVC) is the brainchild of the Fraunhofer Heinrich Hertz Institute, the same group that developed the famous MP3 audio format, as well as AVC and HEVC. Also referred to as Future Video Coding (FVC) — seriously, how many names can we use for the same technology? — VVC can reduce the bandwidth needed for a given video by a further 50% over HEVC.
In an example provided by Fraunhofer, using HEVC, you’d need 10GB of data to transmit a 90-minute 4K video. With VVC, only 5GB of data are required to achieve the same quality.
Fraunhofer HHI developed VVC with the support of several leading electronics companies, including Apple, Ericsson, Intel, Huawei, Microsoft, Qualcomm, and Sony. The standard was finalized on July 7, 2020.
AV1
Running in parallel to the development of VVC is AV1, a codec being developed by the Alliance for Open Media (AOM), a nonprofit organization whose members include Amazon, Apple, Facebook, Google, Samsung, and perhaps most notably, Netflix.
AV1 reportedly offers an average of 30% savings in bandwidth over HEVC — but potentially as much as 40%.
Why would we need AV1 if VVC is more efficient? Well, one big reason is money. AV1 is royalty-free, which means anyone is free to use it without paying a dime. VVC, on the other hand, requires a paid license.
If the cost savings implied by being royalty-free weren’t already enough of an advantage for AV1, it has another ace up its sleeve: Netflix has already started to use AV1 in a limited capacity when streaming to Android devices. When the biggest force in streaming video backs your technology, that’s a good sign it will be widely adopted by others.
Despite its advantages, the path forward for AV1 might not be without hiccups. Even though AOM has declared the codec to be royalty-free, there’s some question as to whether or not it has the ability to make that declaration.
Sisvel, an entity that owns or holds the rights to many technology patents, claims that AV1 is not royalty-free and has set up a site that sells AV1 licenses. The argument is that AV1 was built on top of some of Sisvel’s intellectual property (IP) — as well as the IP of other companies that have joined with Sisvel — and therefore cannot be free from licensing fees.
Given that a poorly executed and fragmented licensing program contributed to the poor uptake of HEVC, AV1 could face similar hurdles if AOM and Sisvel can’t iron out their differences.
MPEG-5 EVC
Finally, the Moving Pictures Expert Group (MPEG) — the same folks who created the MPEG-2 standard that is used for all ATSC digital TV broadcasts — has recently finalized its next-generation codec, MPEG-5 Essential Video Coding (EVC).
Huawei, Qualcomm, and Samsung have already approved it, and have committed to announcing their licensing terms within the next two years. Like VVC and AV1, EVC promises greater efficiency over HEVC — up to 25% less bandwidth needed — but EVC takes a different road to get there. Instead of creating an entirely new video coding system, it applies new coding techniques to existing codecs like AVC, HEVC, and even AV1.
What does this all mean for me?
For the moment … nothing. Sorry, we know this was a lot of information to process for such a small payoff, but here’s a look at why we don’t expect these new codecs to make an impact anytime soon.
Even though it is now 17 years old, H.264 AVC still has a huge chunk of the video codec market: 82% as of 2018 according to at least one estimate. HEVC, by contrast, only enjoyed 12% in the same year.
HEVC’s failure to grab a bigger piece of the pie during its seven-year existence has been chalked up mostly to a rat’s nest of licensing terms that some have characterized as complicated and unfairly implemented.
VVC is clearly a big leap forward from a technology point of view, but observers have already pointed out that it could suffer the same fate as HEVC despite Fraunhofer HHI’s commitment to a uniform and transparent licensing model based on the FRAND principle (i.e., fair, reasonable, and non-discriminatory).
Both VVC and AV1 are able to offer their impressive efficiency gains because they rely on computationally heavy algorithms. In other words, any playback device like a smart TV or a streaming media player that is compatible with these new codecs will have to pack some serious computing horsepower — probably way more than the current generation of devices are capable of.
If you’re Netflix or any other provider of streaming or disc-based video, it probably doesn’t make sense to roll out wide support of a new codec until a significant number of your customers can take advantage of it.
The future is bright
Let’s leave on a positive note. Despite the financial and technological hurdles facing these new video codecs, once the dust settles and they begin to be deployed, it’s going to be a very good thing for folks who want to enjoy 4K, 8K, HDR, or whatever the next big thing in video turns out to be.
The biggest benefit to more efficient codecs is that they can deliver the same high-quality picture we’re used to, but with lower bandwidth.
If your home internet package doesn’t come with unlimited data, this means you’ll be able to stream twice as many 4K movies before you hit your cap. Once 8K movies become mainstream, they won’t be the huge drain on network capacity (and your data plan) that they would be if you watched them with today’s codecs.
And as great as that is for those at home, it’s even more important for mobile users who typically have far more restrictive data allowances.
It’s also possible that the use of these new codecs could bring down streaming media subscription prices. That might be overly optimistic, but consider this: If Netflix switched its entire library of content to videos that had been encoded using VVC, it would not only reduce its network bandwidth by half, it would reduce its storage needs by half too.
Will it pass those savings on to its customers? It would be nice if it did … after all, why shouldn’t better video compression make our monthly bills smaller too?
Updated July 31, 2020: Removed references to AV1 as an open-source technology, clarified that AV1 is considered royalty-free by the Alliance for Open Media and that Sisvel has established a licensing program for AV1.
