Extremist groups have begun experimenting with the use of artificial intelligence, particularly generative AI, to create a host of new propaganda. Experts now worry that the groups’ increasing use of generative artificial intelligence tools will reverse the work Big Tech has done in recent years to prevent their content from entering the internet.
"Our biggest concern is that if terrorists start using generative AI to manipulate images at scale, that could very well destroy hash sharing as a solution," Adam Hadley, executive director of TechAgainstTerrorism, told WIRED. "This is a huge risk."
Big Tech platforms have been working for years to create databases of known violent extremist content, so-called hash databases, which are shared across platforms to quickly and automatically remove such content from the internet. But according to Hadley, his colleagues are now discovering about 5,000 cases of AI-generated content every week. They include images shared in recent weeks by groups linked to Hezbollah and Hamas that appear aimed at influencing the narrative surrounding the Israel-Hamas war.
"In six months or so, the possibility that [they] are manipulating images to crack hashing algorithms is really concerning," Hadley said. "The tech community has done such a good job of building automated technologies that terrorists may well start using genetic AI to circumvent the work that's already been done."
Other examples discovered by TechTerrorism researchers in recent months include a neo-Nazi messaging channel sharing AI-generated images created using racist and anti-Semitic prompts pasted into apps on the Google Play Store; and a far-right individual creating a "memory war guide" advising others how to use AI-generated imagery tools to create extremist memos. "Islamic State" released a technical support guide on how to safely use AI-generated tools; a pro-Islamic State archiving service user claimed to have used an AI-based automatic speech recognition (ASR) system to transcribe Islamic State's Arabic propaganda; and a pro-Al Qaeda media outlet published several posters with images that were most likely created using an AI-generated platform.
In addition to detailing the threat posed by generative AI tools that can tweak images, TechTerrorism has published a new report listing other ways genetic AI tools could be used to help extremist groups. These include the ability to quickly and easily convert promotional content into multiple languages using automated translation tools, or the ability to create personalized messages at scale to boost online recruiting efforts. But Hadley believes that artificial intelligence also provides an opportunity for us to stay one step ahead of extremist groups and use technology to pre-empt attacks.
"We will work with Microsoft to investigate whether there are ways to use our data archives to create a genetic AI detection system to counter the emerging threat of genetic AI being used on a large scale for terrorist content," Hadley said. "We believe genetic AI can be used to defend against hostile uses of genetic AI."
The partnership was announced today ahead of the Christchurch Call Leaders' Summit in Paris. The Christchurch Call for Leaders Summit is a campaign to rid the internet of terrorist and extremist content.
"The use of digital platforms to spread violent extremist content is an urgent issue with real-world consequences," Microsoft Vice Chairman and President Brad Smith said in a statement. "By combining technology's counterterrorism capabilities with artificial intelligence, we hope to help create a safer world both online and offline."
While companies like Microsoft, Google, and Facebook all have their own AI research divisions and are likely already deploying their own resources to tackle the problem, the new initiative will ultimately help companies that cannot tackle these issues on their own.
"This is especially important for smaller platforms that don't have their own AI research centers," Hadley said. "Even now, with hash databases, smaller platforms can be inundated with this content."
The threat of AI-generated content is not limited to extremist groups. Last month, the Internet Watch Foundation, a British non-profit organization dedicated to eradicating child exploitation content from the Internet, released a report detailing the increasing number of child sexual abuse material (CSAM) created by artificial intelligence tools on the dark web.
Researchers found that in just one month, more than 20,000 AI-generated images were posted to a dark web CSAM forum, 11,108 of which were judged by IWF researchers to be the most likely to be criminal. As IWF researchers wrote in the report: "These artificial intelligence images can be so convincing that they are indistinguishable from real images."