Browsed by
Tag: quantizer

Distributed AV1 Video Encoding

Distributed AV1 Video Encoding

Hi everyone,

Yesterday, I began working on writing code to encode video files to AV1 in a distributed computing fashion using my private BOINC server. I wrote some PHP code and developed a Windows Forms software that will submit the video files to the distributed computing server so that it can send jobs to connected machines.

The encoding software I’m using is rav1e. This software is currently single-threaded, which means that encoding videos will take some time, especially when using the –speed 0 flag, which is the slowest mode available nut gives the best quality.

My solution at this time consists on splitting the video files into 1-second segments and then launching a rav1e encoding task for each part, each in one CPU thread. This significantly speeds up the encoding process because it can simultaneously encode several parts depending on the CPU being used. For example, if we split a video into a 1-second segment, and you have an AMD Ryzen 7 2700X CPU, this means that you can process 16 seconds at a time.

Preparing the input video file

As I mentioned at the start, I wrote a Windows Forms software that will handle the video submission process. The software will first call ffmpeg to split the video into 1-second segments, extract the audio file and encode it to Opus using the desired bitrate, and then it will submit the 1-second segment video files to the BOINC server, generating a task for each file:

av1_task_submitter

I’m submitting the video files using a quantizer setting of 175 and a speed of 0.

Once the tasks are generated, comes the processing.

AV1 encoding:

The BOINC client is the software responsible for contacting the server to retrieve the splitted files and initiate the tasks. It will download the rav1e executable from the BOINC server, the input video file, and will start the encoder. This process takes about 2 hours and 30 minutes on an AMD Ryzen 7 2700X CPU while for my Intel i7-3610QM and 4700MQ it took more than 3 hours. Still, the advantage is that I get to encode multiple segments at once, speeding the encoding of the source video significantly.

Here, you can see the encoding tasks from 4 of my machines:

rav1e BOINC Tasks
rav1e BOINC Tasks

Result files:

Once the encoding tasks are finished, they are sent back to the BOINC server which will mark the task as completed and will move the results to a defined folder:

.ifv segments
.ifv segments

Merging the files:

The final step is to merge the result files and also merge the audio file. For this, I wrote a simple bash script that calls ffmpeg’s concat function and then merges the concatenated file with the audio to generate one final .webm file. WebM is s container compatible with AV1 video files as well as Opus audio tracks:

Script to merge the .ifv files into the final .webm file
Script to merge the .ifv files into the final .webm file

The script takes 3 arguments. The first one is the list of files to concatenate, the second is the Opus audio file and the third and final one is the name for the output file.

When we run it, we should see the following at the end:

The merging script after running
The merging script after running

The final file:

We can see the final encoded .webm file at the location we typed when we ran the above script:

The final .webm file
The final .webm file

We can also check the file using MediaInfo:

Mediainfo
Mediainfo

And that’s it! BOINC is really useful in this case as I can encode multiple files and the server will handle all of the processing. The only manual part is where we run the merging script to generate the output file. However, it really speeds encoding by a lot. This is, until rav1e becomes multithreading, at which point, he BOINC server can still be used by changing the app to a multi-threading one.

Hope you enjoyed reading this post!

rav1e AV1 encoding tests – 10/15/2018

rav1e AV1 encoding tests – 10/15/2018

Hi everyone,

Today, I’ve been performing some AV1 encoding tests using the rav1e encoder.

You need an AV1-compatible web browser to see the videos below.

The following videos were encoded using speed 10 and different quantizer settings to compare the quality. rav1e is in constant development so I’ll be performing lots of tests. The following uses the appveyor build from yesterday. I’ve managed to install the dependencies needed to build myself rav1e when new commits comes out, so expect my own builds and more video tests from me!

The original video is a Mario Kart 8 Deluxe clip that I captured using an AverMedia U3 capture card and using OBS with the H264 lossless profile. This video was then converted to y4m using ffmpeg and lastly processed using rav1e.exe.

–quantizer 200 -s 10

–quantizer 175 -s 10

–quantizer 160 -s 10

–quantizer 150 -s 10

Notice that the lower the quantizer setting, the better the footage is. These videos were all using the speed 10 quality setting. I’m currently encoding using the default speed of 3. I’ve tried using speed 0 too but it is insanely slow!!! I’ll try to run an encoding to test but it will take some time. rav1e.exe is currently single-threaded.