OpenAI said it wants to incorporate public input on how to ensure its future AI models are "aligned with human values." To this end, the artificial intelligence startup announced today that it is forming a new "Collective Alignment" team of researchers and engineers to create a system that collects public opinion on the behavior of its models and "encodes" it into OpenAI's products and services.
"We will continue to work with external advisors and funding teams, including conducting pilots to incorporate...prototypes into our model guidance," OpenAI wrote in a blog post. "We are recruiting ... research engineers from diverse technical backgrounds to help us do this work together."
The CollectiveAlignment team is an outgrowth of a public program launched by OpenAI last May to fund experiments in establishing "democratic processes" for deciding which rules artificial intelligence systems should follow. When OpenAI first launched the program, it said its goal was to fund individuals, teams, and organizations to develop proof-of-concepts that answer questions about safeguarding and governance in artificial intelligence.
In today's blog post, OpenAI reviews work from funders that includes a video chat interface, a crowdsourced auditing platform for AI models, and "a method for mapping beliefs to dimensions that can be used to fine-tune model behavior." This morning, all code used in the grantees' work was made public, along with brief summaries and high-level takeaways from each proposal.
OpenAI seeks to separate the program from its commercial interests. But that’s a bit hard to swallow, given OpenAI CEO Sam Altman’s criticism of regulation in the EU and elsewhere. Altman, along with OpenAI president Greg Brockman and chief scientist Ilya Sutskever, have repeatedly argued that AI is innovating so fast that we can’t expect existing institutions to adequately control the technology, necessitating the need to crowdsource the work.
Some OpenAI competitors, including Meta, have accused OpenAI (among others) of trying to ensure "regulation of the AI industry" by lobbying against open AI R&D, which OpenAI denies and will likely point to the grant program (and the CollectiveAlignment team) as an example of its "openness."
Regardless, OpenAI is coming under increasing scrutiny from policymakers, and in the UK it is facing an investigation into its relationship with close partner and investor Microsoft Corp. The startup recently sought to reduce its data privacy regulatory risks in the EU, using a Dublin-based subsidiary to undermine the ability of some EU privacy regulators to act unilaterally.
No doubt partly to reassure regulators, OpenAI announced yesterday that it is working with a number of organizations to try to limit the ways in which its technology can be used to sway or influence elections through malicious means. The startup’s efforts include making AI-generated images generated using its tools more visible and developing ways to identify generated content even after the images have been modified.