Table of Contents

• AV1-vs-HEVC
• Introduction
• Test environment
• Hardware
• Software
• Sample creation
• HEVC encoding
• AV1 encoding
• HEVC results
• AV1 results
• Looking for candidates
• Conclusion

AV1-vs-HEVC

Introduction

In this project, the objective is to assess the performance of the AV1 codec in comparison to the HEVC codec. HEVC has gained widespread acceptance and usage, supplanting the AVC codec, whose sole remaining advantage lies in its dramatically high compatibility. The intended use case here is home cinema. While file size is a consideration, image quality takes precedence. If, over the course of the tests, it becomes evident that the results achievable with AV1 lead to smaller files but at the expense of lower image quality, this would be a decisive criterion against its adoption.

The primary motivation for transitioning from HEVC to AV1 can be succinctly summarized with the following quote from Wikipedia:

Like VP9, but unlike H.264 (AVC) and H.265 (HEVC), AV1 has a royalty-free licensing model that does not hinder adoption in open-source projects.

To conduct the tests, initially, 24 test samples were provided, closely resembling real cinematic content (very closely), yet, of course, free from any copyright concerns and presented in BluRay quality. A test course was devised for these samples to navigate. It is worth mentioning that all tests are conducted with a bit depth of 10 and the pixel format "yuv420p10le".

The goal of the tests is to determine the parameters for the AV1 encoder (SVT-AV1) that yield comparable or even superior results to the HEVC encoder (x265). The evaluation incorporates the following metrics:

• Time required for encoding
• File size compared to the source
• Video quality assessed through the VMAF score

Test environment

All tests were conducted on fairly average consumer-grade hardware:

Hardware

• AMD Ryzen 7 3700X @ factory clock speed
• 32 GiB DDR4 RAM @ 3600MHz

Software

• Arch Linux
• Kernel: 6.6.8-arch1-1
• ffmpeg: 2:6.1-3
• svt-av1: 1.8.0-1
• x265: 3.5-3

ffmpeg -i input.mkv -an -c:v libx265 -preset slow -crf 21 -x265-params "aq-mode=1:qcomp=0.7:vbv-bufsize=9000:vbv-maxrate=9000:no-sao=1:no-strong-intra-smoothing=1:keyint=240:min-keyint=24" -pix_fmt yuv420p10le -y output.mkv

Sample creation

As the source files were extensive movie files and only the video track was relevant for this test, they were stripped of all unnecessary tracks and reduced to a five-minute excerpt (starting from the timestamp 00:30:00 to avoid any kind of intros).

ffmpeg -ss 00:30:00 -i input.mkv -t 00:05:00 -vcodec copy -an -sn -dn -map_chapters -1 -y sample01.mkv

HEVC encoding

Firstly, the samples undergo encoding in HEVC, utilizing the following parameters, which have proven effective in numerous practical applications:

ffmpeg -i input.mkv -an -c:v libx265 -preset slow -crf 21 -x265-params "aq-mode=1:qcomp=0.7:vbv-bufsize=9000:vbv-maxrate=9000:no-sao=1:no-strong-intra-smoothing=1:keyint=240:min-keyint=24" -pix_fmt yuv420p10le -y output.mkv

AV1 encoding

Following that, all samples will be encoded using the AV1 codec. In terms of presets, the initial approach will adhere to the recommendations provided by the Alliance for Open Media:

Presets between 4 and 6 offer what many people consider a reasonable trade-off between quality and encoding time.

For the selection of CRF (Constant Rate Factor) values, we begin with the default setting in HandBrake - 30. To gain a clearer perspective, the value is then iteratively decreased by 1 until reaching a value of 10.

This results in the following command:

ffmpeg -i input.mkv -an -c:v libsvtav1 -preset <PRESET> -crf <CRF> -svtav1-params tune=0 -pix_fmt yuv420p10le -g 245 -y output.mkv

HEVC results

As mentioned above, all samples were initially encoded with HEVC. Now we have a baseline that we should aim to at least meet, if not surpass:

Metric	Mean	Median
Encoding Time	632 sec	603 sec
Filesize Percentage	16.36 pct	13.02 pct
VMAF Score	87.94 pct	92.03 pct

AV1 results

Below are the results of all Preset-CRF combinations in a comprehensive table:

Preset	CRF	Encoding Duration (% of HEVC, Mean)	Encoding Duration (% of HEVC, Median)	Filesize (% of sample, Mean)	Filesize (% of sample, Median)	VMAF score (as %, Mean)	VMAF score (as %, Median)
4	10	245.86 pct	239.07 pct	43.6 pct	32.78 pct	89.3 pct	93.03 pct
4	11	232.04 pct	218.68 pct	37.91 pct	28.11 pct	89.18 pct	92.87 pct
4	12	219.84 pct	206.97 pct	33.33 pct	24.46 pct	89.07 pct	92.73 pct
4	13	207.88 pct	197.28 pct	29.23 pct	21.43 pct	88.95 pct	92.58 pct
4	14	197.81 pct	189.86 pct	25.82 pct	19.13 pct	88.82 pct	92.44 pct
4	15	189.6 pct	181.38 pct	23.06 pct	17.14 pct	88.7 pct	92.31 pct
4	16	181.49 pct	174.23 pct	20.52 pct	15.79 pct	88.57 pct	92.17 pct
4	17	174.94 pct	169.99 pct	18.53 pct	14.42 pct	88.45 pct	92.07 pct
4	18	170.21 pct	167.05 pct	16.82 pct	13.56 pct	88.34 pct	91.97 pct
4	19	164.46 pct	164.8 pct	15.19 pct	12.71 pct	88.21 pct	91.88 pct
4	20	154.19 pct	155.1 pct	13.8 pct	12.04 pct	88.09 pct	91.79 pct
4	21	151.72 pct	153.83 pct	12.8 pct	11.52 pct	87.99 pct	91.73 pct
4	22	149.27 pct	150.66 pct	11.66 pct	10.82 pct	87.81 pct	91.66 pct
4	23	146.89 pct	147.33 pct	10.71 pct	10.28 pct	87.76 pct	91.59 pct
4	24	143.7 pct	142.33 pct	9.66 pct	9.6 pct	87.61 pct	91.51 pct
4	25	139.27 pct	136.57 pct	8.52 pct	8.43 pct	87.42 pct	91.41 pct
4	26	135.39 pct	133.79 pct	7.61 pct	7.58 pct	87.24 pct	91.31 pct
4	27	130.99 pct	129.51 pct	6.86 pct	6.89 pct	87.06 pct	91.22 pct
4	28	127.6 pct	125.21 pct	6.26 pct	6.38 pct	86.89 pct	91.13 pct
4	29	124.03 pct	120.86 pct	5.71 pct	5.88 pct	86.71 pct	91.03 pct
4	30	120.65 pct	116.5 pct	5.28 pct	5.52 pct	86.54 pct	90.94 pct
5	10	99.18 pct	96.81 pct	40.36 pct	28.44 pct	89.16 pct	92.94 pct
5	11	95.47 pct	93.03 pct	35.42 pct	25.04 pct	89.05 pct	92.8 pct
5	12	91.88 pct	91.2 pct	31.23 pct	22.35 pct	88.92 pct	92.66 pct
5	13	88.84 pct	87.65 pct	27.71 pct	20.2 pct	88.81 pct	92.53 pct
5	14	86.25 pct	85.03 pct	24.69 pct	18.44 pct	88.69 pct	92.41 pct
5	15	83.47 pct	81.8 pct	22.05 pct	16.47 pct	88.56 pct	92.27 pct
5	16	80.91 pct	78.84 pct	19.8 pct	14.89 pct	88.44 pct	92.14 pct
5	17	78.63 pct	76.36 pct	17.92 pct	13.95 pct	88.31 pct	92.02 pct
5	18	77.0 pct	75.22 pct	16.26 pct	13.09 pct	88.19 pct	91.91 pct
5	19	75.25 pct	74.32 pct	14.88 pct	12.4 pct	88.08 pct	91.83 pct
5	20	66.89 pct	66.54 pct	13.68 pct	11.85 pct	87.95 pct	91.73 pct
5	21	66.16 pct	65.55 pct	12.62 pct	11.31 pct	87.85 pct	91.66 pct
5	22	65.07 pct	64.12 pct	11.65 pct	10.77 pct	87.73 pct	91.58 pct
5	23	64.59 pct	63.41 pct	10.81 pct	10.18 pct	87.63 pct	91.53 pct
5	24	63.71 pct	62.48 pct	9.91 pct	9.31 pct	87.5 pct	91.45 pct
5	25	62.31 pct	61.05 pct	8.67 pct	8.23 pct	87.3 pct	91.35 pct
5	26	61.16 pct	59.69 pct	7.76 pct	7.46 pct	87.12 pct	91.26 pct
5	27	60.03 pct	57.58 pct	7.03 pct	6.84 pct	86.93 pct	91.16 pct
5	28	59.29 pct	55.95 pct	6.43 pct	6.32 pct	86.76 pct	91.07 pct
5	29	58.31 pct	54.92 pct	5.93 pct	5.96 pct	86.59 pct	90.99 pct
5	30	58.13 pct	54.92 pct	5.44 pct	5.6 pct	86.45 pct	90.97 pct
6	10	72.86 pct	71.36 pct	44.88 pct	31.55 pct	89.23 pct	93.05 pct
6	11	69.82 pct	67.28 pct	38.84 pct	27.29 pct	89.1 pct	92.89 pct
6	12	66.99 pct	64.9 pct	34.07 pct	24.18 pct	88.98 pct	92.73 pct
6	13	64.73 pct	62.62 pct	30.2 pct	21.9 pct	88.86 pct	92.59 pct
6	14	62.76 pct	60.43 pct	27.04 pct	19.99 pct	88.75 pct	92.46 pct
6	15	61.11 pct	58.85 pct	24.37 pct	17.63 pct	88.64 pct	92.34 pct
6	16	59.3 pct	56.95 pct	21.82 pct	16.1 pct	88.5 pct	92.2 pct
6	17	57.65 pct	55.41 pct	19.75 pct	14.67 pct	88.38 pct	92.07 pct
6	18	56.61 pct	55.04 pct	17.94 pct	13.62 pct	88.26 pct	91.96 pct
6	19	55.33 pct	54.14 pct	16.41 pct	12.86 pct	88.14 pct	91.87 pct
6	20	49.21 pct	47.51 pct	15.03 pct	12.29 pct	88.01 pct	91.76 pct
6	21	48.4 pct	46.66 pct	13.86 pct	11.71 pct	87.9 pct	91.69 pct
6	22	47.86 pct	46.02 pct	12.79 pct	11.15 pct	87.79 pct	91.61 pct
6	23	47.39 pct	45.38 pct	11.88 pct	10.7 pct	87.68 pct	91.53 pct
6	24	46.74 pct	44.8 pct	10.93 pct	10.18 pct	87.56 pct	91.46 pct
6	25	45.87 pct	43.8 pct	9.78 pct	9.25 pct	87.39 pct	91.37 pct
6	26	45.09 pct	42.88 pct	8.75 pct	8.23 pct	87.21 pct	91.25 pct
6	27	44.13 pct	42.01 pct	7.85 pct	7.47 pct	87.01 pct	91.16 pct
6	28	43.46 pct	41.46 pct	7.05 pct	6.83 pct	86.82 pct	91.06 pct
6	29	42.65 pct	40.48 pct	6.44 pct	6.3 pct	86.63 pct	90.95 pct
6	30	42.19 pct	39.84 pct	5.87 pct	5.81 pct	86.48 pct	90.92 pct

Furthermore, there are three graphics that illustrate the results more effectively. It becomes evident, for instance, that as the CRF value decreases (across all three presets), the spread in filesizes (mean vs. median) increases.

Looking for candidates

After collecting all the results, an analysis was conducted to determine which AV1 Preset-CRF combinations are at least on par with HEVC. The criteria for this assessment were straightforward and are presented here in pseudocode:

filesize_mean_av1 <= filesize_mean_hevc
filesize_median_av1 <= filesize_median_hevc
vmaf_score_mean_av1 >= vmaf_score_mean_hevc
vmaf_score_median_av1 >= vmaf_score_median_hevc

It turned out that not a single Preset-CRF combination met all the criteria, and in this specific scenario, AV1 is not on par with HEVC. Only when allowing a 20% margin in file size, there is one (!) Preset-CRF combination in AV1 (Preset 4 - CRF 17) whose VMAF score can match HEVC. However, with these settings, the average encoding time also increases significantly by 75%.

Conclusion

The results, with their raw numbers, clearly indicate that the AV1 codec is currently not suitable for the intended purpose mentioned in the introduction. Among all combinations of preset and CRF, there wasn't a single one (!) that could compete with the results of HEVC. While the differences between the results of HEVC and AV1 are not drastic (although a 75% increase in encoding time is noteworthy), one must simply weigh the importance of different criteria. Whether the primary use case is the web or home cinema plays a significant role in this decision. Since the initial focus of the test was on finding the better candidate for home cinema, HEVC must be considered the superior option.