Abstract:
Sony recently submitted a new artificial intelligence technology patent, which proposes to use AI to predict the next frame of the game before the graphics card completes rendering, thereby reducing screen lag and trying to avoid the additional delay problems caused by some existing frame generation technologies. According to the patent description, the system can generate a predicted picture in advance based on the most recently rendered frame and the player's latest controller input information, and display it to the player before the next frame is officially rendered.
This technology mainly targets a flaw in the current partial frame generation scheme. Many existing solutions require obtaining two completed frames before they can calculate and generate the interpolated frame between them, and the process of waiting for the second frame to complete introduces additional delays. Sony's new method attempts to skip this waiting phase and directly use existing pictures to predict future pictures instead of interpolating between two frames.
Theoretically, this method is expected to maintain smoother picture output when the rendering load increases and the frame rate drops. However, the patent document does not provide specific performance test results, so its actual impact on latency improvement and final picture quality remains to be verified.
From a technical implementation perspective, Sony has designed a jointly trained encoder and decoder architecture. The encoder is responsible for converting the rendered image into a compact numerical representation, an embedding vector; the decoder then generates a predicted image based on this data.
While the game is running, the system will work in sync with the graphics rendering pipeline. When the hardware is rendering the next frame, the AI processes the previous frame at the same time and sends the prediction results into the display buffer. This way, the predicted frame can be displayed before the actual rendered frame is completed, filling the rendering gap.
The patent also mentions that the model can use motion vectors and the latest controller input data to improve prediction accuracy. Motion vectors can reflect the moving direction and speed of objects in the image, while key operations can reflect the player's latest behavioral intentions. Both can provide the AI with more information than the previous frame alone.
The training phase uses continuous video frames as the data source. The neural network first starts training with random weights, receives one frame of image as input and tries to predict the next frame, then compares the prediction results with the real subsequent frames, measures the error through the loss value, and continuously optimizes the model.
According to the patent description, the system will continuously detect whether the loss value reaches the set threshold during the training process. Since the continuous video itself can provide real reference images, the entire training process does not require manual data annotation. Additionally, the decoder outputs more color channels than the encoder receives, meaning the system will have the ability to generate multiple predicted frames at once in the future.
In addition to the AI solution, Sony also proposed a non-artificial intelligence alternative. This scheme generates synthetic frames by moving pixel blocks based on motion information without using trained encoder and decoder models. The patent document also claims that these technologies are expected to avoid reducing picture quality while maintaining a fixed frame output rate, but it is still unknown whether the generated frames can truly achieve visual effects similar to native rendering.
Industry insiders believe that the key test of this technology is whether AI can accurately generate predicted images at a fast enough speed and fill the display vacancies in a timely manner when rendering performance declines. The patent shows a feasible idea, but its actual effect still needs to be verified by future product or technology demonstrations.
Comments