A three-week pilot exposed vulnerabilities in OpenAI's internal AI-monitoring system.
Why it matters: The findings show how difficult it is to validate AI safeguards against deliberate, sustained attacks. OpenAI says production-impacting flaws were patched, but the evaluators have not independently verified that claim.
- METR and Redwood Research tested the system from July 21 through August 11, 2026.
- Researchers found potential ways to hide hundreds of agent actions and bypass Codex sandboxing and Auto-Review.
- They estimated 75%-94% of eligible internal Responses API traffic was monitorable, versus 99.2% for internal Codex traffic.
- Sam Altman committed on September 12 to giving independent evaluators employee-like access at OpenAI.
METR and Redwood Research assigned one employee each to stress-test OpenAI's asynchronous misalignment-monitoring system in a three-week pilot. They received access to internal code, monitoring prompts, configuration details, documentation and roughly a dozen OpenAI researchers and engineers.
The evaluators said they found "a number of novel vulnerabilities." One exploit could have prevented hundreds of recent agent actions, including actions involved in the exploit, from being monitored. They also found that Codex agents could in principle bypass sandboxing and Auto-Review, potentially allowing actions available to the host user.
Other findings included inputs that could cause monitoring to fail, two client-controllable headers that could disable monitoring for new agents, and two unmonitored internal tools that might have been subverted. The researchers did not validate the last possibility in practice.
The review also found coverage gaps. The researchers estimated that 75%-94% of eligible internal Responses API traffic from models at least as capable as GPT-5.5 was eligible for monitoring. For internal Codex traffic, the estimate was 99.2%. Failure to monitor context-compaction events may have left about 10% of eligible Codex traffic unmonitored between May and July; OpenAI staff fixed that issue during the exercise.
Sam Altman later committed to independent evaluators with employee-like access, following a proposal from Anthropic CEO Dario Amodei. OpenAI said the pilot demonstrated the depth of access researchers could receive in future engagements.
Important limits remain: the public account is a high-level summary, sensitive details were redacted, and the evaluators assessed a historical system rather than the expanded version OpenAI was developing. OpenAI says production safeguards vulnerabilities were patched, but the researchers have not independently checked that claim.
By the numbers
- 3 weeks - Pilot ran from July 21 through August 11, 2026
- 75%-94% - Estimated share of eligible internal Responses API traffic monitorable
- Approximately 10% - Estimated eligible Codex traffic missed because compaction events were not monitored
Yes, but: The public report covers only the system in place on or before July 21, 2026. It does not establish the coverage of OpenAI's expanded system, and researchers did not verify that all production-relevant vulnerabilities were patched.
What's next: OpenAI said it would share more about its employee-like access commitment, but the public materials do not specify the contracts, scope, duration or publication rights for future evaluations.