Concerns about artificial intelligence often revolve around issues such as misinformation or the potential for the technology to escape human control. However, arguably the more real concern today is how governments are leveraging AI and their institutional understanding (or lack thereof) of its shortcomings. The UK government, for example, appears to have embraced the technology at a pace that might be considered hasty and potentially unsafe.
According to the Guardian, multiple British government agencies have begun to use artificial intelligence to significantly impact the daily lives of ordinary people. The technology now plays a role in a variety of procedures, from arrests and marriage licenses to welfare payments.
Even before artificial intelligence became a widely discussed trend, the use of facial recognition systems by police has been controversial. Critics have long warned of its potential inaccuracies, especially when analyzing darker-skinned subjects. This inaccuracy has even led to wrongful detentions in the past. Despite being aware of these shortcomings, London's Metropolitan Police continues to use facial recognition and make modifications that could harm the technology.
The UK's National Physical Laboratory said the system generally maintains low error rates at default settings. However, if the Metropolitan Police were to reduce its sensitivity in order to identify suspects more quickly it would lead to more false alarms. As a result, the system becomes less accurate for black people, being five times less accurate than it is for white people.
Additionally, AI-based tools used by governments to approve benefits and marriage licenses have shown a tendency to discriminate against applicants from certain countries. One MP highlighted the many instances in recent years where benefits have been inexplicably suspended, leaving individuals on the verge of eviction and extreme poverty. The suspected underlying issue is the Department for Work and Pensions' (DWP) system for detecting benefit fraud, which relies in part on artificial intelligence.
Even without substantial evidence of fraud, the tool disproportionately flagged Bulgarian nationals. The DWP insists the system does not take nationality into account. However, they admit to not fully grasping the inner workings of AI, understanding that systems have limited ability to control bias and worrying that bad actors could manipulate the system.
Likewise, the Home Office is facing the challenge of an AI-driven tool designed to identify fake marriages. While the system streamlined the marriage license approval process, an internal review found a large number of false positives, particularly for applicants from Greece, Albania, Bulgaria and Romania.
There may be other oversights in the government's deployment of AI, but it is difficult to pinpoint them without transparent data from relevant departments.
Misunderstandings about the limitations of artificial intelligence have caused serious incidents within other government and legal agencies. Earlier this year, a U.S. lawyer attempted to use ChatGPT to cite cases filed in federal court, only to discover that all of them had been fabricated by the chatbot. These cases increasingly demonstrate that the real risks of AI may stem more from human misuse than from the technology itself.