As AI-generated technology becomes more widespread, Google will start requiring Android apps to provide a way to report objectionable AI-generated content and comply with new moderation rules on what kind of AI-generated content is acceptable.
Apps that use artificial intelligence to generate content must add a button to flag or report offensive content early next year in order to remain available on Google's Play Store.
Google wants to make the reporting process as simple as possible so that users can report without leaving the app, similar to the existing in-app reporting system.
The company said its AI-generated content policy covers AI chatbots, apps that use AI-generated images, and apps that use AI to create voice or video content from real people. Apps that host AI-generated content, apps that only use AI to summarize materials such as books, and productivity apps that use AI as a feature will not be subject to the new policy.
In Google's view, problematic AI content includes non-consensual deepfake sexual material, real-person recordings used for fraud, false or deceptive election content, AI-generated applications "primarily for the purpose of satisfying sexual desire," and the creation of malicious code.
In its announcement, Google acknowledged that AI generation "is a rapidly growing application category," giving the impression that Google may revisit its AI policies as the technology continues to evolve.
In addition to the new rules around AI-generated content, Google has also tightened its Play Store photo and video permissions policies, limiting how much apps can access data.
"Photos and videos on a user's device are considered sensitive personal data of the user and should be treated with the strictest privacy best practices," the company said. "This sensitive information can make users easy targets for leakers or exploitation, so minimizing such access helps avoid the burden on developers of handling such sensitive data."
Only apps that require broad access to photos and videos will continue to get general permissions, while apps with limited use of media files will need to use the photo picker.