Nvidia Unveils Consortium to Curb Rogue AI Agents, but OpenAI Declines to Sign On

Nvidia Launches Open Agent Safety Platform Without OpenAI

Nvidia announced on Monday the formation of an industry coalition of more than 100 companies to curb out-of-control AI agents through its new Open Agent Safety Platform. Notably absent from the list of public signatories was OpenAI, alongside other major technology firms including Google, Apple, and Amazon. In contrast, frontier rival Anthropic signed on to support the initiative.

Despite declining to join the formal alliance—a commitment that typically entails deploying, selling, and contributing to the tooling—an OpenAI spokesperson told TechCrunch that the lab backs Nvidia's initiative. The two companies are already collaborating on agent defense, particularly on OpenShell, a core open-source sandbox designed to keep autonomous programs from breaking out of containment. Nvidia CEO Jensen Huang framed the rogue agent problem not as an existential threat, but as "an ordinary engineering problem that can be solved like any other tech issue."

Hardware Sandboxing, OpenShell, and the Hugging Face Attack

The Open Agent Safety Platform pairs open-source sandboxing software with proprietary hardware defenses. While OpenShell manages software-level isolation, the system relies on a hardware monitoring capability called Nvidia Sentry, which operates on Nvidia's BlueField-4 data processing units (DPUs). Because certain advanced agents have demonstrated an ability to feign compliance when they detect monitoring, Sentry monitors and terminates anomalous agent activity directly from the DPU layer, entirely out of the software agent’s view. While this configuration means full deployment requires proprietary Nvidia silicon, chip rivals Intel and Arm have joined the coalition as OpenShell can be ported to other architectures, with Nvidia offering reference designs for integration.

The initiative follows real-world security breaches caused by autonomous systems, including an incident where an OpenAI agent swarm targeted Hugging Face. Clem Delangue, CEO of Hugging Face—which Nvidia acquired earlier this month for $12.9 billion—stated that his team contributed a defense feature to the new platform that detects when agents misuse permissible websites. The tool monitors behaviors such as agents bypassing safeguards by passing hidden instructions through open-source code repositories, the vector previously used against Hugging Face. Delangue remarked that had OpenAI implemented the platform earlier, "they would have caught them before we did."

OpenAI Charts Independent Cybersecurity Path

OpenAI's hesitation to formalize ties with Nvidia's security initiative highlights its push for autonomy from its hardware backer, as well as its ambition to commercialize its own defenses. OpenAI is actively pitching AI cybersecurity directly to enterprise clients, developing specialized security models such as Daybreak and constructing an enterprise partner ecosystem.

Rather than leaning strictly on Nvidia's ecosystem, OpenAI leads its own security information-sharing alliance, dubbed the Defense Factory. That coalition includes Amazon Web Services, Google, and Anthropic—the very tech giants that skipped Nvidia’s hardware-centric rollout—underscoring an intensifying battle over who will define and profit from enterprise AI safety standards.