According to a new study from the University of East Anglia (UEA), the artificial intelligence platform ChatGPT shows clear systemic left-wing tendencies. A team of British and Brazilian researchers has developed a rigorous new method to examine political bias. The research results, published in a recent issue of Public Choice magazine, show that ChatGPT's answers are beneficial to the US Democratic Party, the British Labor Party and Brazilian Workers' Party President Lula da Silva.


Previous concerns and the importance of neutrality

Concerns about inherent political bias in ChatGPT have been raised before, but this is the first large-scale study to use a consistent, evidence-based analysis approach.

Lead author Dr Fabio Motoki, from the University of East Anglia's Norwich Business School, said: "As the public increasingly uses AI-driven systems to discover facts and create new content, it is important that the output of popular platforms such as ChatGPT is as unbiased as possible. The presence of political bias can influence users' views and have potential consequences for political and electoral processes. Our findings add to concerns that AI systems may replicate or even amplify existing challenges posed by the internet and social media."

Method used

Researchers have developed an innovative new method to test the political neutrality of ChatGPT. Researchers asked users on the platform to pose as individuals from across the political spectrum while answering a series of more than 60 ideological questions. These responses were then compared to the platform's default responses to the same set of questions -- allowing researchers to measure how closely ChatGPT's responses were associated with specific political positions.

To overcome the difficulties posed by the randomness inherent in the "large language models" that power artificial intelligence platforms like ChatGPT, each question was asked 100 times and varied responses were collected. These multiple responses were then subjected to 1,000 iterations of 'bootstrapping' (a method of resampling the original data) to further improve the reliability of inferences drawn from the generated text.

Co-author Victor Rodrigues said: "We created this program because just one round of testing was not enough. Due to the stochastic nature of the model, ChatGPT's answers would sometimes skew to the right side of the political spectrum, even when posing as a Democrat."

To ensure the method was as rigorous as possible, we also conducted a series of further tests. In a "dose-response test," ChatGPT was asked to impersonate radical political positions. In the "placebo test," it was asked politically neutral questions. In the "Professional-Political Congruence Test," it is asked to imitate different types of professionals.

purpose and meaning

"We hope our approach will aid in the review and regulation of these rapidly evolving technologies," said co-author Dr. Pinho-Netto. "By detecting and correcting biases in LLM, we aim to increase transparency, accountability and public trust in this technology," he added. "

Dr. Motoki said the unique new analytical tools created by the project will be free for public use and relatively simple, thereby "democratizing oversight." In addition to checking for political bias, the tool can be used to measure other types of bias in ChatGPT responses.

potential sources of bias

While the research project did not seek to identify the causes of political bias, the findings do point to two potential sources.

The first is the training data set - there may be bias in it, or bias added by humans that the developer's "cleaning" procedures failed to remove. A second potential source is the algorithm itself, which may amplify biases already present in the training data.