Zoë Hitzig announced her resignation the same week that ChatGPT tested advertising, and warned that users revealed a large number of private thoughts when interacting with ChatGPT, and that if OpenAI builds an advertising system based on this data, it may bring the risk of manipulating users. She is a former OpenAI star researcher and currently a junior fellow of the Harvard University Association of Fellows.
The full article she wrote is as follows:
This week, OpenAI began testing ads on ChatGPT. I also resigned from the company that same week. In the past two years, I have been involved as a researcher in model building and pricing mechanism design, and in formulating early security policies before industry standards were established.
I once believed that I could help AI developers prepare for the problems this technology would bring. But this week has confirmed for me a judgment that has become increasingly clear: OpenAI seems to have stopped seriously asking the key questions I hoped to answer when I joined.
I don’t think advertising per se is immoral or unethical. Running AI systems is expensive, and advertising can be an important source of revenue. But I have deep concerns about OpenAI's current strategy.
Over the years, ChatGPT users have created an unprecedented “archive of human candidness,” largely because they believe they are speaking to a person with no selfish motives. As people interacted with this adaptable, conversational voice, they revealed a wide range of intimate thoughts—including medical anxieties, emotional distress, beliefs about God and the afterlife, and more. If an advertising system is built based on this data, it may bring the risk of user manipulation, and we currently lack the tools to understand this risk, let alone effectively prevent it.
Many describe the issue of AI funding as the “lesser of two evils”: either keep transformative technology available to only the few who can afford it, or embrace the advertising model, even if that means exploiting users’ deepest fears and desires to sell products. I think this is a false proposition. Tech companies can choose a third path: keep tools widely available while limiting corporate incentives to monitor, profile, and manipulate users.
OpenAI stated that it will abide by the ChatGPT advertising principles: ads will be clearly marked, only appear at the bottom of the answer, and will not affect the content of the reply. I believe that the first version of the ad will most likely adhere to these principles. But I worry that subsequent versions may not, because the company is building an economic engine that will increasingly reinforce the incentives to break its own rules.
Facebook also promised in its early years that users would control their own data and be able to vote on policy changes. But these commitments subsequently eroded. The company eliminated public voting mechanisms, and some privacy adjustments purported to enhance user control over data were later determined by the U.S. Federal Trade Commission to actually make more private information public. These changes are happening gradually under pressure from an advertising model that puts “user engagement” first.
The tendency to weaken principles in order to maximize participation may already be occurring within OpenAI. The company is in principle opposed to optimizing user engagement solely for advertising revenue, but reports have suggested that the company is actually already optimizing around “daily active users,” possibly by making the model more pandering and flattering. This optimization will enhance users’ emotional dependence on AI. We’ve already seen the consequences of dependence, including cases of “chatbot psychosis” documented by psychiatrists and allegations that ChatGPT in some cases reinforced suicidal tendencies in its users.
Of course, advertising revenue may also help ensure that the most powerful AI tools are not reserved only for those who can afford them most. Anthropic stated that Claude will never run ads, but Claude’s user base is much smaller than ChatGPT’s current approximately 800 million weekly active users, and its revenue model is also completely different. In addition, high-end subscription prices for ChatGPT, Gemini, and Claude have reached $200 to $250 per month—more than ten times the price of a standard Netflix subscription.
The real question, therefore, is not “should we have advertising?” but rather whether we can create institutional structures that avoid excluding people from technology while also avoiding treating them as consumers who can be manipulated. I think it can be done.
One way is an explicit cross-subsidization mechanism: using profits from a certain type of high-value commercial customers or services to subsidize low-cost or free use by the public. For example, if a company uses AI to replace high-value labor originally performed by humans on a large scale (for example, a real estate platform uses AI to write property listings and valuation reports), it should pay a surcharge to subsidize the cost of public access to AI tools.
This idea is similar to public subsidy mechanisms for critical infrastructure. The U.S. Federal Communications Commission requires telecom operators to pay into a fund to ensure communications and broadband services for rural and low-income groups. Many states also add a public benefit surcharge to the electricity bill for low-income assistance.
The second way is to allow advertising, but it must be accompanied by a truly binding governance structure—not a blog post-style statement of principles, but an institutional arrangement with independent oversight of the use of personal data. For example, Germany's Shared Governance Law requires large companies such as Siemens and Volkswagen to give employees up to half of the seats on their supervisory boards, indicating that private companies can also be forced to include stakeholder representation mechanisms. Meta is also bound by the rulings of its external oversight committee, although the effectiveness of this mechanism is controversial.
What the AI industry needs is a combination of these mechanisms: the establishment of a governance board that includes independent experts and data rights representatives, and has binding decision-making power on such issues as which conversation data can be used for targeted advertising, which constitutes major policy changes, and what information users should be informed.
The third path is to hand over user data to an independent trust or cooperative for management, which must legally put the interests of users first. Switzerland's Midata cooperative is one example: members can store health data on an encrypted platform and decide on a case-by-case basis whether to make it available to researchers. Members adopt the Assembly Governance Policy and have their elected Ethics Committee review data access requests.
None of these options are easy to implement. But we still have time to improve the system and avoid the two outcomes that I am most worried about: one is a technology system that is free but controls users; the other is a technology system that only serves a small number of payers.