The Meta Oversight Committee recently issued a decision calling on companies to tighten and clarify regulatory rules as soon as possible to deal with the rapid proliferation of AI-generated content and deep fake videos in high-risk situations such as wars, disasters, and elections. The agency pointed out that Meta currently has obvious shortcomings in labeling and identifying large-scale AI-generated content, especially during times of conflict and crisis, and this flaw may mislead the public.
The criticism stems from a fake video that appeared on the Meta platform last year. The AI-generated video purports to show footage of building damage in the Israeli city of Haifa and was posted by a user in the Philippines posing as a news source. Many users reported the video, but the platform neither conducted an internal review nor triggered a third-party fact-checking mechanism. The video had been online for a long time but was not marked as "high-risk AI content." It did not attract attention until the oversight committee intervened.
The Oversight Committee's role is to improve Meta's treatment of users and the community globally. The committee believes that this case exposed systemic problems with Meta’s AI content governance system on platforms such as Facebook, Instagram, and Threads, which relies too much on users’ voluntary disclosure or internal “upgrade” processes and cannot match the current reality of AI content spreading at an alarming speed and scale, especially during wars, disasters, and highly politically sensitive events.
Currently, Meta’s labeling of AI-generated content depends largely on whether the creator self-declares the use of AI, or whether the content is escalated internally to a special review process. The Oversight Committee pointed out that this approach is far from sufficient to deal with the proliferation of deep fakes and synthetic media on the Internet, and it is difficult to promptly remind users how the content is generated and potentially misleading.
In its latest recommendations, the Oversight Board asked Meta to develop a separate set of community standards for AI-generated content rather than continue to rely on a fragmented patchwork of misinformation policy responses. The committee advocates that the rules for AI-generated images, videos, and audios should be unified into a clear framework so that platforms can make more consistent and transparent interventions in high-risk scenarios.
The committee also recommended that Meta use the “high-risk AI” label more frequently to prominently label content that could affect public safety, public opinion, and elections. At the same time, companies need to further improve the capabilities of automated detection systems to cover multiple media forms such as images, videos, and audio, and clearly explain to users the specific penalties that those who fail to disclose digital synthesis or tampered content will face.
In terms of technical foundations, the Oversight Committee named the Content Credentials industry framework and believed that Meta should adopt this mechanism more actively and uniformly. This framework aims to improve content traceability and transparency by attaching metadata to content to indicate the source of the material and whether AI tools are used. The committee is concerned that Meta is currently inconsistent in its enforcement of such standards, including in its labeling of content generated by its own AI tools.
The Oversight Board has previously criticized Meta's "manipulation of media" rules as being too confusing and too narrow in scope, focusing too much on "whether AI is used" rather than "whether the content is deceptive." Under pressure, Meta adjusted the relevant labeling system and replaced the original "Made with AI (made by AI)" label with the broader "AI info (AI information)" label. However, this change has also attracted doubts from the outside world. It is believed that the new label prompt is too general and not eye-catching enough, and users can easily ignore the key information.
On the eve of the inauguration of current US President Donald Trump, Meta CEO Mark Zuckerberg announced that the company's existing third-party fact-checking partnership had become "too politically biased" and undermined users' trust in the platform, and would be replaced by a Community Notes mechanism. This adjustment means that the platform will further rely on spontaneous labeling and error correction by the user community in identifying and correcting misinformation. In the context of the rapid proliferation of deep fake and AI content, this strategy has also triggered continued debate about its effectiveness.