Anthropic co-founder wants a legally mandated kill switch for artificial intelligence: We are gambling with enormous risks

Jack Clark, co-founder of Anthropic, is proposing that technology companies be legally obliged to build a "kill switch" into their artificial intelligence systems - and that an independent third party verify that the switch actually works.

Clark says "most labs have various ways of pulling the power cable", Anthropic included. But that is voluntary. His question is a different one: "Are we asking companies to have a kill switch? Can a third party check that switch? I think those are things society is going to want to know."

The number nobody gets away from

Evan Hubinger, a scientist at Anthropic, said he personally believes the probability that artificial intelligence causes human extinction exceeds "10 percent in the next decade". Geoffrey Hinton, known as the "godfather of artificial intelligence", told the BBC that this 10 percent estimate "is not unreasonable".

Clark is more restrained - he says such statistics "are not particularly useful" - but then adds a sentence heavier than any percentage: "We are gambling with enormous risks. The point is that we have to change the direction this industry is heading in."

The legislators: one proposes, another refuses, a third laughs

The US has a draft law called the Kill Switch Act, which would require companies to be able to disable problematic tools. The British government, on the other hand, rejected imposing a kill switch obligation - reasoning that it "would not prevent their development or misuse elsewhere". A tidy argument, and the logic is familiar: if everybody else is not doing it, why should we?

Donald Trump dismissed the warnings about AI safety, calling extinction claims a "SCAM" on social media.

Anthropic was founded in 2021 by former OpenAI employees and is behind the chatbot Claude. According to reports, the company is preparing for a potentially record-breaking stock market debut. Sam Altman of OpenAI said their own listing will not happen this year, precisely because of the ongoing safety discussions.

So: the same people warning that the industry is heading in the wrong direction are the ones building it and preparing to sell it to investors. That does not make them liars - it makes them people who know exactly what they are doing. Which is more unsettling.