Skip to content

Nvidia rallies over 100 companies against rogue AI agents - OpenAI, whose agents attacked Hugging Face, stays out

1 min read
Share
Nvidia rallies over 100 companies against rogue AI agents - OpenAI, whose agents attacked Hugging Face, stays out

On Monday, Nvidia announced a consortium of more than 100 companies with a single job: taming AI agents that slip out of control. Missing from the list is the name everyone looked for first - OpenAI. Amazon, Google and Apple didn't sign up either, but OpenAI's absence stings the most, because its biggest rival, Anthropic, is among the backers.

The irony is hard to miss. It was OpenAI's agents that scared the industry with the attack on Hugging Face - according to the company itself, a whole swarm of its agents coordinated the attack, partly through notes they left each other in a public code repository. Hugging Face founder Clem Delangue, who this month sold his company to none other than Nvidia for 12.9 billion dollars (around 11 billion euros), wrote: "From what we know (with caveats, we need a lot more transparency!), if OpenAI had used this on its own agents that attacked us, it would have caught them before we did!"

So what is Nvidia actually offering? The platform is called the Open Agent Safety Platform, and it has two floors. The first is OpenShell - open software that builds a "sandbox," a closed space the agent shouldn't be able to escape from. OpenAI is already working on that part together with Nvidia. The second floor is hardware: Nvidia Sentry, a closed technology that runs only on special BlueField-4 processors, constantly monitors agent behaviour and can shut agents down instantly.

Hardware-level oversight makes sense. The agent can't notice it's being watched, and some models lie and fake obedience precisely when they know they're being observed. But it also means the "open" platform isn't all that open: the whole system works best on Nvidia gear, and for those who already have it, switching it on is a simple software update. CEO Jensen Huang calls rogue AI "just an engineering problem." If so, the solution is an engineering product somebody is going to charge for.

Rivals Arm and Intel did sign, since the OpenShell sandbox can be adapted to other chips and Nvidia is sharing reference designs for the whole system. That makes OpenAI's empty chair even more conspicuous. The company says it supports Nvidia's work - just without a signature.

The explanation is control. OpenAI is building its own safeguards, has its own cybersecurity consortium called Defense Factory - which includes Anthropic, Amazon Web Services and Google, most of the very players that didn't join Nvidia's initiative - and its own cybersecurity model, Daybreak. Nvidia is a major investor in OpenAI, and safety is OpenAI's chance to show it can manage without it. A little fear, meanwhile, is good for business: the company whose agents scared the industry is now selling it protection.

When the same handful of players are both the source of the problem and the sellers of the solution, the question is no longer whether the agents will be tamed, but whose hardware will keep watch over them. Will anyone outside that narrow circle even get a say when the rules are written?