Anthropic has embedded six researchers from METR, a nonprofit AI evaluator, within its internal teams to assess AI risks. This unprecedented access allows METR to publish assessments without Anthropic's approval, though redactions may apply.
TL;DR
- Anthropic grants METR researchers access to its internal AI systems for independent risk assessments, with publication rights.
- This move follows Anthropic's commitment to stronger independent evaluations under the White House Accord on Super Intelligence.
- METR's access is the most extensive among external evaluators, but publication rights have limits.
What happened
Anthropic has given six METR employees access to its internal tools, systems, hardware, and office space, similar to its own risk-assessment teams. These researchers are encouraged to share information with the METR team as they would with colleagues assessing AI alignment risks.
METR can publish assessments of Anthropic's risk levels, incidents, and practices without seeking the company's approval. However, Anthropic can require redactions involving customer data, attorney-client privileged material, and model architectures.
The six embedded employees work under enforceable agreements, and other METR employees do not have access to the embedded team's work. This arrangement places METR at the highest level of access for external evaluators, combining employee-like access with publication rights.
Why it matters
This expanded access follows Anthropic's September commitment to stronger independent evaluations and its voluntary pledge under the White House Accord on Super Intelligence. The accord encourages signatory companies to work with independent external assessors to examine whether their internal AI safeguards operate as intended.
For developers and startups, this move sets a precedent for how AI companies can collaborate with external evaluators to enhance transparency and trust in AI safety. It also highlights the importance of independent assessments in the AI industry.
However, METR has acknowledged the limits of voluntary risk assessments, stating that companies could refuse requests for internal information. This raises questions about the extent of access and the independence of external evaluators.
Key facts
- Anthropic has embedded six METR researchers within its internal teams for AI risk assessments.
- METR can publish assessments without Anthropic's approval, but redactions may apply for customer data, attorney-client privileged material, and model architectures.
- The six embedded employees work under enforceable agreements, and other METR employees do not have access to the embedded team's work.
- Anthropic has worked with METR on reviews of its Risk Reports for about a year.
- Anthropic is also working with SecureBio and Accenture's specialist AI business, Faculty, but those partnerships have not reached METR's level of access.
- METR does not accept payments or donations from frontier AI developers, aiming to disclose limitations that could affect how readers interpret its findings.
- The White House Accord on Super Intelligence encourages signatory companies to work with independent external assessors to examine internal AI safeguards.
Context
This move is part of a broader trend in the AI industry towards greater transparency and independent evaluation. As AI models become more powerful and complex, there is a growing need for robust safety measures and independent assessments to ensure their responsible development and deployment.
Anthropic's collaboration with METR sets a precedent for how AI companies can engage with external evaluators. It also highlights the challenges and limitations of voluntary risk assessments, as noted by METR.
The AI industry is increasingly recognizing the importance of independent evaluations in building trust and ensuring the safe use of AI technologies. This collaboration between Anthropic and METR is a significant step in that direction.
