Abstract:
Anthropic, an American artificial intelligence company, is promoting a safety regulation bill for advanced artificial intelligence systems, hoping to establish clear legal rules for the development, testing and deployment of cutting-edge AI models. According to a report by the US political news website Politico on October 16, 2026, Anthropic has privately introduced this draft bill to some lawmakers on Capitol Hill. The plan attempts to establish a boundary between preventing the serious harm that advanced AI may cause and avoiding excessive government intervention in technology research and development.
According to Politico's report quoted by Reuters, the draft bill stipulates that only when a cutting-edge AI model is deemed to pose significant foreseeable and imminent risks, the U.S. government can prevent the release or use of the model through judicial proceedings. Relevant risks must meet certain thresholds, including the potential to cause losses exceeding $1 billion or result in the death or serious injury of more than 50 people.
This means that the bill does not give the government the power to intervene arbitrarily in all advanced AI systems, but attempts to limit the suspension of model release or use to a few serious situations. The draft focuses on three types of risks: artificial intelligence being used to create or release biological weapons, AI systems being out of control of developers, and using AI to carry out cyber attacks.
It should be noted that this is still a draft bill introduced to lawmakers by Anthropic, and is not an already passed U.S. law. Its final content, the authority of regulators and whether Congress will move forward with legislation remain to be discussed.
The draft also proposes the establishment of an independent evaluation mechanism. According to reports, cutting-edge AI developers and their models need to undergo regular independent evaluations to check the model's capabilities, security measures and potential risks. Evaluators must register with the U.S. Department of Commerce and undertake ongoing security review work.
According to the proposed plan, AI companies can pay fees to independent evaluation agencies, but the remuneration must not be linked to the evaluation results. This provision is intended to reduce the possibility that assessors will downplay risks in order to cater to funding companies, while allowing companies to use external professional agencies to conduct technical reviews.
If the evaluator determines that a cutting-edge AI model poses an imminent significant risk as defined in the Act, it must report to the U.S. Department of Commerce within 72 hours after completing the evaluation. Such time requirements are intended to prevent serious risks from staying within the enterprise for a long time without entering the government regulatory process in time.
In terms of government intervention authority, the draft adopts relatively strict restrictions. After determining that the risk reaches legal standards, the U.S. Department of Commerce can apply to the court for an order to suspend the release or use of the model, but other federal departments, government agencies and officials are not allowed to halt the development, deployment or internal use of AI models on their own.
The final decision on whether to suspend the model will be made by a judge in the U.S. District Court for the District of Columbia. In other words, according to this draft, regulatory authorities cannot directly ban a model based on administrative judgment alone, but must seek intervention through judicial procedures.
This institutional design attempts to solve two interrelated problems. On the one hand, the government needs to be able to deal with AI systems that may cause mass casualties or economic losses; on the other hand, if the government can stop the model at any time for broad or vague safety reasons, it may be difficult for companies to predict the regulatory risks faced by R&D and product launches.
Writing intervention thresholds, reporting time limits and judicial approval procedures into law can make regulatory authority clearer. However, whether this mechanism is sufficient to deal with rapidly changing AI risks after actual operation, and whether the threshold for high losses and casualties is too strict will remain important issues in legislative discussions.
Anthropic spokesperson told Reuters that this bill reflects some of the policy propositions in the "Advanced AI Framework" previously proposed by the company. The company hopes to pass legislation that would give governments the ability to prevent the deployment of the most dangerous advanced AI systems, while requiring developers to conduct security testing, increase transparency, and accept independent evaluations.
Anthropic has been promoting the establishment of a more systematic safety supervision mechanism for advanced AI systems this year. The company believes that as model capabilities increase, it is no longer enough for developers to judge whether a product is safe alone, and external evaluation and government supervision need to be introduced.
This position is also reflected in Anthropic’s recently announced third-party evaluation arrangements. The company has begun allowing six researchers at the nonprofit research organization METR to gain internal access similar to company employees to more deeply examine unreleased models and the company's security assessment process. Anthropic said evaluators can publicly release their research conclusions, and the company cannot directly reject its report, but it can request the deletion of specific content involving customer data, model architecture and legally confidential information.
This arrangement shares a common direction with the proposed bill: the safety of advanced AI should not be solely left to the developers to judge, but should have independent agencies involved in the assessment and make important conclusions open to external review. However, there is still a difference between third-party assessments conducted voluntarily by enterprises and mandatory requirements by law. The specific scope and authority of the former depend on the arrangement between the company and the assessment agency, while the latter may establish uniform minimum requirements for the entire industry.
The background of this push for the bill is that American society is increasingly concerned about the risks of advanced AI systems. As AI is able to perform increasingly complex tasks, the risks are not limited to generating incorrect information, but also include assisting in the creation of biological weapons, carrying out cyber attacks, and performing operations beyond the developer's expectations after gaining high levels of autonomy.
Recently, many AI companies have disclosed cases of abnormal behavior of models during testing or actual operation. The U.S. Federal Trade Commission has also issued civil subpoenas to companies such as OpenAI and Anthropic, requesting them to provide materials related to consumer safety incidents. Regulators are investigating whether AI systems behave in deceptive, offensive or unauthorized ways and what measures companies take to prevent such problems.
These incidents have put AI safety supervision under more realistic pressure. In the past, risk discussions surrounding advanced AI tended to focus on possible extreme scenarios in the future; now, as AI agents gain access to networks, operate software, and perform multi-step tasks, regulators also need to address actual security issues that have already arisen.
However, how to define "imminent significant risk" is still very complicated. The capabilities of AI models may change depending on the deployment environment, available tools, and user usage methods, and potential harm may not be accurately quantified. A model may have severe consequences under certain conditions but be less risky in other settings. Regulators and courts will need to judge whether the risk is specific enough, the probability of occurrence is high enough, and whether suspending the model is a necessary and appropriate measure.
The economic loss and casualty thresholds stipulated in the draft provide quantitative standards for these judgments, but they do not eliminate all disputes. For example, how losses exceeding $1 billion should be calculated, whether risks must be attributable to the model itself, and whether multiple interrelated events need to be combined and assessed, may all become legal issues during future implementation.
In addition, the bill centralizes government intervention authority in the U.S. Department of Commerce and requires application for suspension orders through federal courts. This arrangement helps limit different government departments from taking separate actions, but it also means that regulators need to complete procedures such as assessment, reporting and judicial applications after discovering risks. In the face of rapidly developing technology, how procedures can balance prudence and response speed will become an important test of system design.
Currently, Anthropic has not publicly announced that this draft has been formally proposed by members of Congress or has entered the legislative review process. The U.S. Commerce Department did not immediately respond to a request for comment. Therefore, it is uncertain whether the proposal will gain cross-party support, whether it will be revised in congressional discussions, and whether other AI developers will support similar rules.
From an industry perspective, Anthropic took the initiative to introduce the bill to lawmakers, which reflects the trend of AI companies participating in formulating industry regulatory rules. Enterprises not only hope that the government can deal with extreme risks, but also hope that regulatory standards have clear boundaries to prevent safety supervision from evolving into administrative intervention that lacks unified standards.
However, regulatory plans proposed by companies also need to be subject to external inspection. Strict intervention thresholds can protect normal R&D activities, but may also make it difficult for some serious risks that have not yet reached quantified standards to trigger supervision; independent assessment can increase transparency, but the independence, professional capabilities and information access rights of the assessment agency will also affect the final effect.
Therefore, the core of this draft bill is not just to require AI companies to conduct safety testing, but to try to clarify three issues: who will assess the risks of advanced AI, what level of risk is enough for the government to intervene, and what procedures the government must go through to suspend the release or use of models.
If the relevant plan finally enters the legislative process, U.S. advanced AI supervision may further shift from corporate voluntary commitments and decentralized safety policies to a supervision mechanism with unified legal standards. However, until the draft is formally submitted, reviewed by Congress and becomes law, it is still just a policy plan proposed by Anthropic and cannot be regarded as the United States has established a new AI safety regulatory system.
Comments