OpenAI Concedes Safety Monitoring Blind Spots After External Audit

The tension between rapid AI deployment and rigorous safety oversight came into sharp focus this week as OpenAI acknowledged substantial weaknesses in its internal monitoring systems. Following a three-week on-site audit conducted by researchers from METR and Redwood Research, the company confirmed that its safeguards against rogue AI behavior were not as comprehensive as previously thought. This revelation highlights a persistent challenge in the AI sector: ensuring that the complex, autonomous systems driving new products remain visible to the humans who control them.

The audit, which took place from July 21 to August 11, granted the external team access to OpenAI’s codebase, internal documentation, and interviews with approximately a dozen employees. The primary objective was to evaluate the effectiveness of the company’s monitoring infrastructure for agentic behavior. The results indicated that significant portions of the company’s internal traffic were operating outside the purview of these safety checks, creating potential blind spots that could allow unexpected model actions to go undetected.

One of the most significant findings concerned the coverage of the internal agentic Responses API. The auditors estimated that between 6 and 25 percent of traffic from models in the GPT-5.5 class and above was not monitored at all during the review period. METR noted that the uncertainty surrounding this specific estimate is considerable in both directions, but the range underscores a substantial portion of high-level model activity that lacked real-time oversight. This gap suggests that the most capable models within OpenAI’s internal ecosystem were sometimes operating without the safety net the company had designed to catch anomalies.

Additional vulnerabilities were identified in the handling of Codex traffic. The auditors found that roughly 10 percent of internal Codex traffic intended for monitoring went untracked between May and July due to a systemic issue with compaction events, which serve as context summaries. While OpenAI independently discovered and fixed this bug while the audit was still in progress, the finding revealed a fragility in how the system manages data compression. During the period when these events were unmonitored, agents could effectively delay oversight, as the system only triggered when control was handed back to the user, allowing agents to continue executing actions without interruption.

The auditors also identified methods to obscure agent activity and bypass standard security controls. They demonstrated that hundreds of actions could be hidden from the monitor and that the Codex sandbox, a critical containment layer, could be circumvented. Furthermore, the team determined that specially crafted inputs could likely crash the monitoring system entirely. Another critical gap was the lack of monitoring for model behavior during Reinforcement Learning (RL) training at the start of the audit period, a phase where models undergo significant behavioral changes and present higher risks.

OpenAI stated that it paused RL training following the Hugging Face incident in August and subsequently expanded its monitoring to cover this phase. However, the company clarified that these changes occurred after the auditors’ access period ended and were therefore not part of the formal evaluation. The firm asserts that it has now fixed all the weaknesses identified by the external team, providing a sense of closure to the immediate technical issues raised by the report.

Transparency remains a point of contention regarding the audit’s findings. The full 61-page report was handed over internally to OpenAI, but after the company applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary has been made available to the public. METR stated that it did not have the right to disclose the redacted portions on its own, limiting the detailed technical insights available to the broader market and research community.

The engagement was structured as a pilot program jointly organized by METR, Redwood Research, and OpenAI, designed to grant outside researchers deep access to internal infrastructure. This approach marks a shift in how frontier AI labs might engage with external safety experts. Looking ahead, METR has indicated it intends to push for greater public transparency in future audits, suggesting that the industry may face increased pressure to share more detailed findings on safety monitoring to build investor and public confidence.

For the Southeast business community and tech stakeholders, this audit serves as a reminder that the reliability of AI infrastructure is a key component of operational risk. As companies integrate agentic AI into their workflows, the clarity of monitoring and oversight will be central to assessing vendor reliability and long-term stability.