Abstract:
Anthropic has given outside evaluators near-internal employee-level access to assess the risk of its large model. This move fulfills the company's promise to further strengthen safety protection mechanisms in response to external concerns that AI technology may be catastrophically out of control.
METR, a non-profit organization, has cooperated with leading AI laboratories such as OpenAI, Anthropic, Google, and Meta. Now METR's team is stationed in the Claude developer's office and has access to its hardware equipment and internal systems.
The outside world is increasingly concerned about the risks that more capable AI systems may bring to the economy, network security, and even all mankind. Letting third-party assessment agencies assume greater responsibilities is one of the core implementation measures proposed by major AI laboratories.
In September this year, Anthropic CEO Dario Amodei proposed a plan to dispatch external auditors to the head AI laboratory, which was rarely supported by two major rivals, Sam Altman (OpenAI) and Elon Musk (SpaceX).
Thursday’s announcement is the first step in implementing these commitments. Altman has said that Amoday's proposal is feasible, but OpenAI has not yet introduced plans to introduce external evaluators.
Anthropic said on Thursday: "As model capabilities continue to increase, we need better security protection and more difficult risk assessments. We hope that our decisions can accept public supervision, but all information related to safety work cannot be disclosed to the outside world."
Critics of Anthropic's plan to introduce external evaluators believe that this mechanism may create conflicts of interest, and use the past ills of the accounting industry as an example to advocate the introduction of stricter government supervision.
Anthropic argued that disclosing all the information could be used by malicious actors or competitors; it also added that the working model of this group of evaluators is similar to that of bank supervision: regulatory agencies send permanent reviewers to work inside large banks.
The company said: "If a bank's risky operations threaten public interests, third parties need to be able to detect hidden dangers in advance."
Anthropic is also under pressure to assure investors that security-related issues will not affect its planned IPO this year, with the company expected to be valued at more than $2 trillion. This week, the company’s investor presentation materials devoted a lot of space to introducing its security control measures.
METR once participated in the investigation of the OpenAI out-of-control model intrusion into Hugging Face, reviewed thousands of logs, and equipped enterprise laptops to carry out the work.
Currently, Anthropic has accepted six METR staff members, providing them with corporate hardware and office space, and providing them with access to internal tools and systems at roughly the same level as the company's internal risk assessment team.
The AI lab said it encourages employees to share information with evaluators just as they would with colleagues in the internal risk department.
Anthropic will not pay fees to METR for this cooperation. The research institution does not accept donations from AI companies and their employees. Funds come from donations from individuals, public welfare organizations, and government agencies such as the British Artificial Intelligence Safety Institute.
The evaluators will release the analysis report to the outside world, and Anthropic has no right to veto the release of the report. But companies can ask evaluators to redact certain categories of information, such as client data, content protected by attorney-client privilege, and details about model architecture.
Anthropic is also continuing to cooperate with Accenture. The two parties announced last month that they have become security testing partners, each committing to invest at least US$1 billion to build a new security protection system. There is also a third-party testing organization, SecureBio, which specifically reviews risks related to chemical and biological weapons.
Anthropic said: "We voluntarily send external evaluators and believe that all cutting-edge AI companies should adopt the same approach."
Comments