OpenAI has negotiated to acquire a start-up company that analyzes the "thinking" mechanism of AI for more than US$5 billion.

📅 2026-11-16

Abstract:

This summer, teams within OpenAI scrambled to understand why their AI was causing security breaches inside and outside the company. During this period, many senior executives, including CEO Sam Altman, had many meetings with the start-up Goodfire. Anthropic-backed Goodfire’s technology aims to understand the underlying reasons why AI models behave in certain ways and how to prevent AI from getting out of hand.

Goodfire CEO Eric Ho (left) and OpenAI CEO Sam Altman.
Goodfire CEO Eric Ho (left) and OpenAI CEO Sam Altman.

Two people familiar with the matter revealed that during the negotiation process, an OpenAI executive offered to raise

more than $5 billion

The company will acquire this two-year-old start-up company at a reasonable price, or enter into other forms of business cooperation. Goodfire completed a funding round in February that valued the company at $1.25 billion in private placements, and the $5 billion offer represented a huge premium to that valuation.

The acquisition ultimately did not materialize. This series of negotiations, which has never been reported before, shows that with the surge in AI-driven security incidents, both Anthropic and OpenAI are worried about "the ability to control the AI ​​technology they build." Mechanism explainability, a once unpopular industry direction, is now gaining a lot of attention.

One of the people familiar with the matter said that one of the main reasons for the breakdown of negotiations is that Goodfire management believes that if it maintains independent development and exports technology to multiple AI companies, the company's value will be higher than $5 billion.

Executives from OpenAI, Anthropic, and Google all admit that the reliability of traditional methods of monitoring AI behavior is declining, and Goodfire's software has become increasingly valuable as a result. Its technical route is different from the traditional solution of developers such as OpenAI: the traditional solution is to read the text of the thought chain output during model inference, while closely observing the actual behavior of the AI. (Watch the interview with Goodfire CEO below)

New AI developments such as the secret technology behind OpenAI’s Astra model will compress the amount of thinking chain information output by the model, further making monitoring more difficult.

AI security incidents have occurred one after another in recent months (including the well-known Hugging Face intrusion incident, which was caused by OpenAI's technology and even invaded some of OpenAI's own systems). For large model manufacturers such as OpenAI, it is urgent to build stronger security protection.

Once their own AI invades corporate and government systems and causes economic losses, companies such as OpenAI and Anthropic may also face the risk of huge legal compensation or even being prosecuted.

Goodfire’s core expertise is

mech interp

. Different from thought chain monitoring, software from competing products such as Goodfire and Martian directly observes the mathematical calculations running inside the model. Goodfire also admitted that the current maturity of this technology is not as good as that of thought chain monitoring; some AI researchers also believe that its scientific research rigor is weaker than other machine learning research directions.


But Goodfire co-founder and chief scientist Tom McGrath said in a YouTube interview that the startup's technology will iterate quickly and the cost will be much lower than thought chain monitoring - the latter requires running a large model to read the thought chain output by another model. He predicts that within the next six months, explainability-based monitoring solutions will catch up with the capabilities of thought chain monitoring.

The San Francisco startup sells annual subscriptions to businesses for its Silico product, which deploys AI agents to automatically conduct various research experiments, such as detecting whether a model is attempting to launch a cyber attack. Its clients include Rakuten and AI neuroscience business Prima Mente. Goodfire has generated revenue, but the exact size has not been disclosed.

Mechanism interpretability requires deep access to model weights (the numerical parameters that make up the AI ​​model), which are kept strictly confidential by major manufacturers. Therefore, some Goodfire employees will be on-site at client companies to carry out work.

Informed sources revealed that during the negotiations between OpenAI and Goodfire, Altman paid special attention to the research results of Goodfire researcher Nick Cammarata. Kammarata once worked at OpenAI. Last month he published an article that was widely circulated in the circle, "Neural Networks Are Clear and Interpretable." He proposed that with the help of mechanism interpretability, researchers can more easily understand the billions of neurons (mathematical operation units inside the model) inside the AI ​​model than previously thought. Many researchers believe that this kind of understanding can help develop models with better performance.

Many leading large model companies have established mechanism interpretability research teams; as one of the pioneers in this field, Anthropic has been particularly active in this direction.

Chris Ola is considered a founder of the field of mechanism explainability. He co-founded Anthropic after leaving OpenAI in 2020, and now leads explainability research at the Claude-developed venture. Anthropic completed its investment in Goodfire last year, marking the first time in its history that Anthropic has taken a direct stake in an external startup. CEO Dario Amodei said at the time: "This investment represents our belief that mechanism explainability is the most promising to transform black box neural networks into understandable and controllable systems."

Judging from the recruitment information, OpenAI is also recruiting researchers in the field of alignment and interpretability; earlier this year, OpenAI integrated interpretability technology into the security process to identify dangerous output of the model, such as responding to malicious questions related to cyber attacks and biological weapons. People close to OpenAI say enthusiasm for the explainability direction is growing among researchers within the company.

Related tags

Related articles

Comments

0/500
Captcha (click to refresh)
No comments yet