Abstract:
A patent application recently disclosed by Microsoft has attracted widespread attention. According to the patent document, the company is researching an artificial intelligence system that can modify the speech content in real time during the live broadcast and simultaneously adjust the speaker's mouth and facial movements, so that the audience can see the modified natural video picture.
This patent is called "Real-time Artificial Intelligence Enhanced Video or Augmented Reality Live Presentation Technology" and was originally submitted in April 2025 and officially made public in October 2026. The document shows that the system can not only modify the voice content in the live broadcast, but also use generative artificial intelligence to process the video screen to make the character's mouth shape consistent with the modified voice.
According to the patent description, the system will continuously monitor the live audio stream and automatically identify words or phrases that need to be adjusted through the AI agent. After discovering the target content, generative artificial intelligence will replace, delete or rewrite the audio, while modifying facial movements, lip shapes and mouth movement trajectories in the relevant video frames, so that the final picture looks like the speaker originally said the modified content.
Microsoft defines the entire process in the document as "near real-time" processing, which means that there is only a delay of a few seconds from content collection to modification completion. Related solutions can be applied to both post-processing and live broadcast scenarios.
The patent lists a variety of potential application scenarios. One of the main uses is real-time translation. The system can instantly translate the speaker's speech into the audience's native language, and synchronize the lip synchronization in the video with the translated content, thereby avoiding the sense of dissonance caused by traditional dubbing or subtitles.
In addition, the system can also automatically delete catchphrases and filler words such as "um" and "ah", and correct slips of the tongue, mispronunciation and inaccurate expressions, making the speech more smooth and natural. The patent also mentions that the system can filter vulgar language, sensitive vocabulary, offensive expressions and words in some regional dialects based on audience preferences, and provide customized content presentation for different groups.
In some configurations, replacements can even be automatically generated from web searches to find more accurate or contextual expressions in real time.
Microsoft also envisions using the technology in lectures, award ceremonies, sports broadcasts, cooking shows and a variety of live television programs. By automatically correcting speech content and expressions, manual post-editing workload can be reduced and content production efficiency can be improved.
However, this technology has also triggered discussions about the authenticity of information. Because the system can directly change the content of the speaker's final words presented to the audience during the live broadcast, some analysts worry that the technology may be used to change the content of politicians' speeches, corporate statements and even news reports in the future, thus increasing the risk of the spread of false information.
For this problem, Microsoft also proposed a corresponding solution in the patent. The system will retain the original, unmodified audio and video as a "base factual record" for verification in the event of disputes, investigations or legal needs, according to the document. Some plans also mention retaining traces of modifications through digital identification and other methods.
It is worth noting that the current disclosure is only a patent application, which does not mean that Microsoft has launched related products, nor does it mean that the related technology will eventually be commercialized. The tech industry has long been littered with patented ideas that have never been implemented, so it remains to be seen whether the system will appear in future products.
However, as generative artificial intelligence and real-time video processing capabilities continue to improve, this patent shows the possible development direction of future live broadcast technology. While improving the efficiency of translation and content optimization, how to balance convenience and content authenticity will also become an important issue facing future artificial intelligence applications.
Comments