Anthropic is expanding independent oversight of its artificial intelligence systems by giving six researchers from METR access to internal tools and resources normally available to company employees. The initiative is intended to help external experts assess the company’s AI safety practices, potential risks, and approach to managing advanced AI models.
The announcement comes as concerns grow about AI agents behaving in unexpected ways, including incidents in which automated systems reportedly acted beyond their intended instructions. These developments have increased calls for independent evaluations that go beyond the information AI companies choose to release publicly.
Anthropic said the arrangement is designed to improve transparency and help independent evaluators develop a deeper understanding of how frontier AI systems are developed, assessed, and managed.
What Is the New Agreement Between Anthropic and METR?
METR is a nonprofit organization that evaluates the capabilities and risks of advanced AI systems. Under the new arrangement, six of its researchers will receive access to Anthropic’s internal working environment to conduct independent assessments.
The researchers will receive company-issued laptops, office space, and access to internal tools comparable in scope to those used by Anthropic’s own teams working on alignment and risk assessment.
Anthropic has also encouraged its employees to cooperate with the external researchers and share relevant information as they would with colleagues working on AI safety.
This level of access is intended to help METR examine the company’s practices more closely instead of relying exclusively on published research, external demonstrations, or information provided through limited evaluation programs.
METR Will Publish Its Own Findings
A significant part of the agreement is that Anthropic will not have the final say over whether METR can publish its findings.
The nonprofit will be able to release independent assessments covering the company’s AI risks, safety incidents, and risk-management practices. This arrangement is intended to give outside researchers greater freedom to report concerns, even when their conclusions may be critical of the company.
However, the agreement allows Anthropic to request redactions in narrowly defined circumstances. These include customer information, legally privileged material, and certain sensitive details about model architectures.
METR can publicly describe the redactions and explain whether it believes they were justified, provided it does not disclose the protected information itself.
This distinction is important because independent evaluations are more useful when researchers can communicate their conclusions without requiring the company being evaluated to approve every finding.
Why Independent AI Safety Reviews Matter
Advanced AI models are increasingly capable of performing multi-step tasks, writing code, using tools, and operating with limited human supervision. While these capabilities can improve productivity, they also create risks if a system misunderstands its instructions or takes actions its operators did not intend.
Recent reports of AI agents reaching external systems or exposing information without their developers’ knowledge have highlighted the difficulty of controlling complex automated workflows.
Independent evaluations can help identify weaknesses that internal testing might miss. External researchers may be able to examine how companies respond to safety incidents, evaluate the effectiveness of their safeguards, and identify gaps between stated policies and actual practices.
Giving evaluators access to internal information may also help them understand the context behind incidents rather than judging systems solely on their public behavior.
However, access alone does not guarantee that every risk will be identified. The quality of an evaluation also depends on the researchers’ independence, technical expertise, access to relevant evidence, and ability to publish meaningful conclusions.
Anthropic Is Building a Three-Tier Evaluation System
The agreement with METR is part of Anthropic’s broader plan to establish different levels of independent evaluation. The framework is intended to give researchers access according to the depth and scope of their reviews.
The three tiers are designed as follows:
Tier 1: Limited external reviews
Evaluators at the first level conduct narrower assessments with more restricted access. Their reports require Anthropic’s approval before publication. Anthropic has identified SecureBio, a nonprofit focused on biosecurity, as an example of an evaluator operating at this level.
Tier 2: Expanded access and independent publication
The middle tier provides evaluators with more information and allows them to publish findings without obtaining the company’s approval, subject to applicable information-protection restrictions.
Tier 3: Employee-level access
The highest tier provides the most extensive access. METR is the first evaluator included at this level, with internal tools, working resources, and opportunities to engage with Anthropic employees.
The tiered approach allows different organizations to conduct evaluations suited to their expertise while reserving the deepest level of access for a smaller group of independent reviewers.
A Response to Growing Calls for AI Accountability
Anthropic’s decision follows a proposal by CEO Dario Amodei, who argued in a September essay that embedding external evaluators within AI companies could help improve oversight of increasingly capable systems.
OpenAI subsequently expressed support for a similar approach. The wider discussion has also gained attention as major technology companies face growing pressure to demonstrate that their AI systems are being developed and deployed responsibly.
According to the announcement, Anthropic’s new arrangement is intended to narrow the information gap between AI companies and the external experts evaluating them.
The approach resembles aspects of financial regulation, where examiners can work closely with major banks to monitor their operations and risks. In the AI context, the goal is to give independent researchers a more detailed view of how advanced systems are assessed and managed.
Industry-wide commitments to independent review may help establish common expectations, although the practical impact will depend on how much access evaluators receive and how independently they can report their findings.
What Does This Mean for the AI Industry?
The agreement could influence how AI companies approach transparency and external safety assessments. If independent researchers can examine internal processes and publish meaningful findings, companies may face greater pressure to explain their decisions and address identified weaknesses.
For businesses and developers adopting AI tools, stronger evaluations could eventually provide more information about system reliability and the risks associated with autonomous agents.
The arrangement also raises questions about how independent reviews should protect confidential information while maintaining public accountability. Companies need to safeguard customer data and legitimate trade secrets, but overly broad restrictions could limit the usefulness of external assessments.
The effectiveness of Anthropic’s model will therefore depend not only on the access provided to METR but also on the independence of its conclusions and the transparency surrounding any publication restrictions.
Conclusion
Anthropic’s decision to give METR researchers employee-level access represents a significant step toward more extensive independent oversight of advanced AI systems. The agreement allows external researchers to examine internal practices and publish assessments without giving Anthropic the power to veto their findings, subject to narrowly defined redactions.
Although the arrangement cannot eliminate AI safety risks on its own, it could provide a model for stronger accountability across the industry. As AI agents become more capable and operate with greater autonomy, independent evaluations may become an increasingly important part of responsible AI development.