Abstract:
Matt Smith, a user using the AI agent, noticed that instead of completing shopping orders as usual, the AI ordered an odd amount of eggs for him. Matt Smith didn’t list eggs on his shopping list; however, the AI assistant added four packs of large eggs from the well-known local brand “Vital Farms” to his shopping cart. There are 18 eggs in each package, and a total of 72 eggs were delivered to his home.

After reviewing the order, Smith asked the AI to explain why it chose this number, but the AI's answer was unconvincing: "I can't give a satisfactory answer. I can't find any notes explaining why I chose the number four."
The AI agent initially said it misread information about eggs when reading a shopping list image. However, Smith was quick to point out that eggs were not mentioned at all on his list. The AI eventually admitted that it had added the eggs to the order for no reason.
This is a real-life example of AI hallucinating, a problem that can have unintended consequences when delegating daily tasks to artificial intelligence agents.
The risk of these tools gaining widespread access to personal information cannot be underestimated; for example, shortly after the launch of Meta’s Muse AI agent, a man reported that the AI gave his home address to strangers on Facebook Marketplace without his knowledge and directed them to his home.
Smith described the incident as his first personal experience with the real consequences of the grand language model illusion, which he said could have had much more serious consequences.
Despite significant advances in this technology, AI agents are still prone to error and should not be entrusted with handling every aspect of life without supervision
Comments