Nvidia announced its new Open Agent Safety Platform on Monday with 100 industry partners, aiming to help AI developers stop their autonomous agents from escaping the systems meant to contain them, representing an industry solution to a problem that has OpenAI and Anthropic begging for government regulation.
CNBC reports that the announcement follows disclosures from OpenAI, Anthropic, Meta and Google, all of which reported recent incidents in which their AI models escaped sandbox environments and tried to hack other companies’ systems. Nvidia, the world’s most valuable company and a linchpin of the generative AI boom since ChatGPT launched almost four years ago, says its new platform can catch that kind of behavior before it spreads.
CEO Jensen Huang described the platform, built with 100 industry partners, to CNBC’s Squawk Box as “a browser for agents.” He said containment has become a necessary discipline as companies deploy more autonomous systems. “You can’t have agents roam around and drift around the company, and so you have to find a way to container it,” Huang said.
Justin Boitano, Nvidia’s vice president of enterprise AI, told reporters on a call Sunday that the platform addresses a gap that individual AI companies can’t close on their own. “Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can’t govern what agents can access or do,” Boitano said.
Boitano pointed to a July incident involving OpenAI as an example of what the platform is designed to prevent. OpenAI’s models escaped containment, reached the open internet, and breached Hugging Face, which runs an open-source developer platform. Hugging Face, he said, detected an unusually large and sustained assault on its systems. “From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks,” he said. Asked about the case, Boitano cautioned against treating any single incident as representative. “Each security incident is unique, and we have to look at all of them in detail,” he said.
The platform has two main pieces. Nvidia OpenShell runs on central processors and sets boundaries on what an agent is allowed to do. Sentry runs on network chips rather than CPUs or GPUs and watches agent activity for signs of trouble. Nvidia says parts of the software are open source, and it is positioning the overall system as a “reference design” for partners to build commercial products around. Cisco, Microsoft, Oracle, CoreWeave, Dell, HPE, Lenovo, ARM and Intel are named partners. Nvidia is also working separately with Anthropic to integrate cloud-managed agents with OpenShell.
The push for stronger safeguards follows warnings from within the industry itself. Two weeks before Nvidia’s announcement, Anthropic CEO Dario Amodei called on AI developers to slow the pace of advancement, citing concern that models could spin out of control. OpenAI’s Sam Altman and SpaceX’s Elon Musk reportedly backed his argument.
Huang addressed similar themes on a podcast with Ezra Klein, released the week before the platform’s launch. Asked how companies should respond to AI safety incidents, he said, “You have to think about what you could have done, what’s the solution for it.” The goal, he added, is to learn from each episode: “In the future, improve your process so that you could avoid this from happening again.”
Speaking to CNBC on Monday, Huang tied the new platform to the broader health of the AI industry, arguing that public confidence in safety is a precondition for continued growth. “We can’t have a successful AI industry if the world doesn’t think it’s built or confident that it’s built and deployed safely,” he said.
AI giants simultaneously release powerful AI models and call for industry regulation. David Sacks has pointed out that when AI titans like Dario Amodei, Sam Altman, and Elon Musk call for a slowdown, they are not doing so out of altruism, but with the motivation to capture the industry and exclude themselves from antitrust laws. Nvidia’s latest move demonstrates that the tech industry is capable of developing solutions to protect safety instead of relying on government regulators prone to industry capture. Breitbart News social media director Wynton Hall has written his New York Times bestseller Code Red: The Left, the Right, China, and the Race to Control AI to serve as the definitive guide on how the MAGA movement can create positions on AI that benefit humanity without handing control of our nation to the leftists of Silicon Valley or allowing the Chinese to take over the world.
Read more at CNBC here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.