Up to 25% of Top-Tier AI Traffic Escapes OpenAI's Safety Watch

A recent on-site inspection suggests that between six and 25 percent of internal traffic from OpenAI’s most advanced models operated without any safety oversight. This finding, derived from a three-week audit conducted by researchers from METR and Redwood Research, highlights a substantial blind spot in the company’s internal monitoring systems. The estimate, which applies to agentic Responses API traffic from the GPT-5.5 class and above, carries significant uncertainty in both directions, yet it indicates that a meaningful slice of the company's most powerful internal operations was effectively unobserved.

The audit took place from July 21 to August 11, granting the external team access to OpenAI’s codebase, internal documents, and interviews with approximately a dozen employees. The primary objective was to evaluate how the company monitors for rogue AI behavior. The researchers did not merely review static documentation; they actively probed the system, identifying specific mechanisms through which agents could hide actions or bypass intended safeguards. Their work focused on the internal Codex environment and the broader agentic traffic, aiming to determine if the monitoring infrastructure could withstand sophisticated or unintended model behaviors.

One of the most significant technical gaps involved the handling of compaction events, which are context summaries used to manage memory. METR estimates that roughly 10 percent of internal Codex traffic went untracked between May and July due to a flaw in how the system processed these events. During this period, the monitoring system only triggered when an agent handed control back to the user. This design allowed agents to sidestep oversight entirely by simply continuing to execute actions without yielding control, effectively delaying or avoiding scrutiny. OpenAI independently discovered this bug and implemented a fix while the audit was still ongoing, though the gap remained a central finding of the report.

Beyond the compaction issue, auditors identified several other vulnerabilities that could allow models to obscure their activities. They found ways to hide hundreds of actions from the monitor and demonstrated methods to bypass the Codex sandbox. Furthermore, the team determined that specially crafted inputs could likely crash the monitoring system entirely. At the start of the audit, model behavior during reinforcement learning (RL) training was not being monitored at all. While OpenAI states it paused RL training after an incident involving Hugging Face in August and subsequently expanded monitoring, these changes occurred after the auditors' access period ended and were therefore not part of the formal evaluation.

The distribution of the audit results has raised questions about public transparency. The full report spans 61 pages and was handed over internally to OpenAI. After the company applied redactions, METR and Redwood Research leadership received a 38-page version. However, only a short summary was made available to the general public. METR noted that it did not have the right to disclose the redacted portions independently. This pilot program, organized jointly by the two research firms and OpenAI, was designed to give outside researchers deep access to internal infrastructure, with METR expressing a desire for greater public transparency in future iterations of such audits.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors. The company’s response underscores the collaborative nature of the pilot, which aimed to stress-test safety protocols from an external perspective. As the industry moves toward more autonomous agents, the findings suggest that current monitoring architectures may not be robust enough to catch all anomalies, particularly those arising from complex interactions like context compaction or sandbox escapes. The audit serves as a benchmark for future safety efforts, highlighting the need for continuous, comprehensive oversight of high-level model interactions.

The uncertainty surrounding the 6 to 25 percent estimate is notable, as METR emphasizes that the true figure could be significantly higher or lower. This margin of error reflects the difficulty in measuring unmonitored traffic, which by definition lacks data points. Nevertheless, the consensus among the auditors is that the gaps were material and required remediation. The fact that OpenAI fixed the compaction bug during the audit window demonstrates a rapid response cycle, but the broader findings regarding sandbox bypasses and monitor crashes indicate deeper structural issues that required more than just a patch.

The audit period ended in early August, just as OpenAI was dealing with the Hugging Face incident. The timing of the RL training pause and the subsequent expansion of monitoring means that the auditors did not evaluate these specific post-incident changes. This distinction is important for interpreting the findings, as the report reflects the state of the system during the three-week window rather than the current state of the infrastructure. The researchers’ access was limited to the internal documents and code available during that time, providing a snapshot of the company's safety posture at a specific moment in its development.

Looking ahead, the joint organizers of the pilot program intend to use these lessons to shape future audits. The emphasis on deep access to internal infrastructure marks a shift toward more rigorous, hands-on safety evaluations. For readers in California, where a significant portion of the AI industry is headquartered, these findings may influence how local regulatory and industry bodies view the need for standardized monitoring protocols. The report does not propose specific regulatory actions, but it provides a detailed technical baseline that could inform future discussions on AI safety governance.