Three days after Amazon released Q, its artificial intelligence chatbot, some employees are raising alarms over accuracy and privacy concerns. According to leaked documents obtained by Platformer, Q was "having severe hallucinations and leaking confidential data," including AWS data center locations, internal discount programs and unreleased features. One employee labeled the incident "sev2," meaning it was serious enough that engineers were called in at night and asked to work over the weekend to fix it.
Q's early woes come as Amazon grapples with the perception that Microsoft, Google and other tech companies are surpassing Amazon in the race to build tools and infrastructure to take advantage of artificial intelligence. In September this year, Amazon announced that it would invest $4 billion in artificial intelligence startup Anthropic. On Tuesday, at Amazon Web Services' annual developer conference, Amazon released Q, which is arguably the most high-profile of a series of new artificial intelligence initiatives the company has released this week.
In a statement, Amazon downplayed the importance of employee discussions.
"Some employees are sharing feedback through internal channels and the ticketing system, which is standard practice at Amazon. This feedback did not identify any security issues. We appreciate all the feedback we have received and will continue to adjust Q as it transitions from a preview product to generally available," the spokesperson said.
Q is now available as a free preview. Amazon executives said on stage this week that initially it will be able to answer developer questions about AWS, edit source code and cite sources. It will compete with similar tools from Microsoft and Google, but the price will be lower than the competition, at least initially.
In launching Q, Amazon executives touted it as more secure than consumer-grade tools like ChatGPT.
Amazon Web Services CEO Adam Selipsky told the New York Times that "companies are prohibited from using these artificial intelligence assistants due to security and privacy concerns." In this regard, the New York Times reported that "Q created by Amazon is more secure and private than consumer chatbots."
An internal document on Q's hallucinations and incorrect answers states: "Amazon Q may hallucinate and return harmful or inappropriate answers. For example, Amazon Q may return outdated security information, putting customer accounts at risk. The risks outlined in the document are typical of large language models, all of which return incorrect or inappropriate answers at least some of the time."