Crucially, Wikipedia references are the links to sources that support the information in the online encyclopedia. But sometimes, these references are flawed, pointing to broken websites, incorrect information, or untrusted sources. A study published in Nature Machine Intelligence on October 19 shows that artificial intelligence (AI) can help clean up inaccurate or incomplete reference lists in Wikipedia entries and improve their quality and reliability.
Fabio Petroni and colleagues at London-based SamayaAI have developed a neural network-powered system called "SIDE" that can analyze whether Wikipedia references support relevant claims and suggest better alternatives for those that do not.
"Given that ChatGPT is notorious for errors and illusions about citations, it seems a little ironic to use AI to help process citations," said Noah Giansiracusa, who studies artificial intelligence at Bentley University in Waltham, Massachusetts. "But it's important to remember that AI language models can do much more than chatbots."
SIDE is trained to identify good references using Wikipedia's existing featured articles, which are promoted on the site and receive widespread attention from editors and moderators. It can then identify claims in the page that contain low-quality references through the validation system. It also scans the internet for reputable sources and lists options for replacing bad references.
To test the system, Petroni and his colleagues used SIDE to recommend references for featured articles on Wikipedia that it had never seen before. In nearly 50% of cases, SIDE's preferred reference is already cited in the article. In other cases, it found other references.
When the results of SIDE were shown to a group of Wikipedia test users, 21% preferred references found by the AI, 10% preferred existing references, and 39% had no preference.
Aleksandra Urman, a computational communications scientist at the University of Zurich in Switzerland, said the tool can save time for editors and moderators checking Wikipedia entries for accuracy, but only if it is used correctly. "The system could be useful in flagging citations that might be inappropriate," she said. "The question is what the Wikipedia community would consider most useful."
Ullman points out that Wikipedia users who tested the SIDE system were twice as likely to dislike both citations as they were to like the AI-recommended citation. "That means, in this case, they're still going online to search for relevant citations," she said.