Anthropic has brought six METR employees into its offices to examine AI systems, widening independent access while retaining authority to require redactions of certain sensitive findings.
In an Oct. 15 announcement, Anthropic said it supplied the nonprofit evaluator’s team with company equipment and workspace. The employees can use internal tools with access similar to that of Anthropic’s own risk assessment teams. The company also said it was in early discussions with Hugging Face about evaluation work.
METR may publish what it finds, and Anthropic says it cannot veto the findings. Anthropic can require redactions within predefined categories of sensitive information and decides whether material falls within those categories.
Anthropic says giving outside evaluators deeper access can improve public scrutiny. It has compared the arrangement with resident bank examiners, who work inside the institutions they oversee.
Access alone does not settle whether an evaluation is credible. At an Oct. 15 TechCrunch Disrupt session, Hugging Face Chief Policy Officer Irene Solaiman said reviewers need expertise suited to the systems they assess, TechCrunch reported. She said inconsistent methods make results difficult to compare. Reproducing a test may also require access to a model’s weights.
Solaiman also addressed AI agents, which pair a model with other software to perform tasks. An evaluator who tests only the model may miss the effect of that surrounding software, known as a harness. A better harness could let a weaker model outperform a stronger one, she said. Assessing a complete agent remains expensive, and testing methods are still developing.
The four organizations have also been part of a U.S. discussion about AI oversight. At a Sept. 15 congressional briefing on AI loss-of-control incidents, the House Science, Space, and Technology Committee met with representatives of Anthropic, METR, Hugging Face and OpenAI.
Anthropic has set Dec. 15, 2026, as its target for a proposal on how governments could support embedded evaluators, including their funding, access and independence.
