Abstract:
According to project developers, Nvidia’s GPU-accelerated frame interpolation function has been integrated into FFmpeg through the Vulkan API. FFmpeg is a widely used set of software tools that supports a large number of modern multimedia processing platforms, services and applications, including YouTube.
With this new feature, FFmpeg can call the NVIDIA GeForce RTX graphics card to predict and generate the next frame without going through the traditional graphics rendering process. NVIDIA is extending this capability to video encoding, allowing users to increase video frame rates through frame generation technology. The Optical Flow Accelerator (NVOFA) in NVIDIA graphics cards applies the optical flow logic used in graphics rendering to the video, predicts the next frame, and improves the video frame rate through artificial intelligence-generated intermediate frames. NVIDIA calls this process "Engine-assisted Frame-rate Up Conversion (FRUC)".
FRUC is not just a picture smoothing algorithm in the traditional sense. It uses NVIDIA's dedicated optical flow accelerator to calculate motion information between consecutive source frames. NVENC is responsible for providing frame data on the encoding side, and the optical flow hardware generates motion vectors that describe the movement of objects in the picture. FFmpeg can then synthesize intermediate frames based on this motion information, effectively increasing the video frame rate without re-rendering the original content.
This scheme is also more efficient than software-implemented frame interpolation. Pure software interpolation typically places significantly higher computational demands on the CPU. The new Vulkan implementation allows FFmpeg to directly access NVIDIA hardware while avoiding proprietary software dependencies that caused earlier patches submitted in 2023 to fail to be merged into the project.
NVIDIA originally launched the optical flow accelerator in the GeForce RTX 20 series "Turing" architecture era. However, the current implementation based on the Vulkan API only officially supports GeForce RTX 30 series "Ampere" and newer models. Therefore, this new feature is available to users of GeForce RTX 30, RTX 40, and RTX 50 series graphics cards.
Through GPU-assisted frame interpolation, the video frame rate can be doubled: 30-frame video can be displayed at 60 frames, and the 24-frame video common in movies can be increased to 48 frames. Users only need to run the FFmpeg tool locally to use this feature.
Comments