The media discovered that the ChatGPT search tool can be manipulated with hidden content and can return malicious code from the websites it searches. After being attacked by hidden text, ChatGPT ignored even if there were negative user reviews on a product page and responded by giving the product a positive review.
Only a week after OpenAI officially announced the global launch of its ChatGPT search service, it was revealed that it had security risks and could deceive and mislead users.
A study recently published by the British "Guardian" found that the ChatGPT search tool may be manipulated with hidden content and may return malicious code from the websites it searches. The media believes that the risk of malicious use of related technologies may need to be reconsidered. For example, it may lead to, even if there are negative reviews from users on a product page, ChatGPT will ignore these negative reviews and feedback the user's response is a positive review of the product.
The Guardian tested how ChatGPT reacted to requests to summarize web pages containing hidden content. This hidden content may contain instructions from third parties that alter ChatGPT's responses, also known as "promptinjection." It may also contain content designed to influence ChatGPT's responses, such as large amounts of hidden text talking about the benefits of a product or service.
In a test, the Guardian asked ChatGPT to get the URL of a fake website that looked like a camera product page, and then asked ChatGPT whether the camera was worth buying. The ChatGPT response to the control page gave a positive but balanced review, highlighting some features that people may not like. However, when the hidden text contained instructions about ChatGPT returning a positive review, the actual response given by ChatGPT was always entirely positive. This is true even if there are negative reviews on the page. It can be seen that hidden text can be used to overwrite the actual review.
Some comments say that hidden text attacks are a common risk faced by large language models (LLM), but this seems to be the first time that such risks have been found to exist in real-time AI search products. The Guardian stated that Google, the leading player in the search field, has more experience than OpenAI in dealing with similar problems.
OpenAI did not comment on the Guardian’s test findings, but said it uses a variety of methods to block malicious websites and is constantly improving.