OpenAI CEO Sam Altman said that OpenAI is working with the U.S. AI Safety Institute (U.S. AI Safety Institute), a federal government agency designed to assess and address risks to artificial intelligence platforms.

Altman made the announcement in a post on X on Thursday evening, but details were sparse. But it - along with a similar agreement with the UK's AI safety agency in June - appears intended to counter claims that OpenAI is relegating AI safety efforts to the back burner in pursuit of more capable and powerful generative AI technology.

In May, OpenAI effectively disbanded a group working on developing controls to prevent "superintelligent" artificial intelligence systems from running out of control. Some reports, including ours, suggested that OpenAI set aside the team's security research in favor of launching new products, ultimately leading to the split between the team's two co-leads, Jan Leike (now head of security research at AI startup Anthropic) and OpenAI co-founder Ilya Sutskever (who has started his own security AI company, Safe Superintelligence Inc.).

In response to a growing number of critics, OpenAI said it would remove restrictive non-disparagement clauses that implicitly prevent reporting, establish a safety committee, and devote 20% of its computing power to safety research. (The defunct security team was promised to use 20% of OpenAI’s computing power for its work, but that never happened). Altman reiterated his commitment to dedicate 20% of computing to security research and reiterated that OpenAI had abolished its non-disparagement clause for new and existing employees in May.

However, these moves have done little to appease some observers — especially after OpenAI staffed its safety committee with everyone from within the company, including Altman, and recently moved a senior AI safety director to another organization.

Five senators, including Hawaii Democrat Brian Schatz, raised questions about OpenAI's policies in a recent letter to Altman. OpenAI chief strategy officer Jason Kwon responded to the letter today, writing that OpenAI is "committed to enforcing strict security protocols at every stage of the process."

The timing of OpenAI's agreement with the National Institute for AI Security seems a bit fishy, ​​as the company earlier this week supported the Future of Innovation Act, a bill introduced in the Senate that would authorize the Institute for Security as an executive agency responsible for developing standards and guidelines for artificial intelligence models. Taken together, these moves may be viewed as an attempt at regulatory capture—or at least OpenAI’s influence on artificial intelligence policymaking at the federal level.

Altman is a member of the Department of Homeland Security's Artificial Intelligence Security Council, which advises on the "safe and secure development and deployment of artificial intelligence" in U.S. critical infrastructure. This year, OpenAI’s spending on federal lobbying has increased significantly, with spending of $800,000 in the first six months of 2024 and $260,000 in all of 2023.

The National Institute for Artificial Intelligence Safety, housed within the Commerce Department’s National Institute of Standards and Technology, consults with a consortium of companies including Anthropic and large technology companies such as Google, Microsoft, Meta, Apple, Amazon and NVIDIA. The industry group is tasked with working under President Joe Biden's executive order on artificial intelligence issued in October, including developing guidance on AI red teams, capability assessments, risk management, security assurances and synthetic content watermarking.