The CEO of Anthropic, Dario Amodei, wrote something in a new essay that, coming from a man who sells artificial intelligence, sounds like sabotaging his own business: "We must slow the pace at which we improve the capabilities of models." He added that progress will nonetheless look fast and that the time gained must be used wisely. The question he put to himself is who exactly is in such a hurry.

The occasion is not philosophical. Amodei named two things that pushed him to write - the security breach at OpenAI and HuggingFace, and the fact that models are beginning to improve themselves. Before that, an Anthropic researcher named Jacob Coxon resigned and left with a sentence that is not easily forgotten: that the people at the top of the industry sincerely believe the technology could kill us all before the decade is out. A man who builds the thing walks away from it and says so out loud - that is a data point in itself.

Three proposals, one of which needs permission from the state

The first is internal oversight: independent evaluators from organisations like METR sitting inside the firms and checking whether promised safety behaviour is actually being carried out, with the same access to internal risk assessments. Anthropic says it will do this of its own accord, and wants governments to oblige everyone to do the same. Sam Altman of OpenAI replied that the proposal is a good idea and that they are joining in - "I agree with Dario that we should regulate the pace at the frontier," he wrote. Elon Musk was shorter: "Dario is right."

The second proposal is harder. Leading firms in democratic states would agree on shared safety standards and on limiting the speed of development. Only, an agreement between the largest players in a market about how fast they may operate has another name in law, and Amodei knows it - which is why he asks for a narrow exemption from antitrust rules, with a government mediating without participating. Read that twice. Competitors are asking for legal cover so they can sit at the same table.

The third is global, including talks with authoritarian governments. Amodei admits expectations there are low, but believes that at least for the obviously dangerous applications - biological weapons, for instance - some common language can be found. For Chinese competition he proposes an older solution: restricted access to chips and a fight against model copying, which in his view would give America a three-to-five-year lead.

Who benefits from the braking

The journalist Brian Merchant and others in the industry immediately raised the question that hangs over the whole thing - is this safety, or is it regulatory capture? When two firms already at the top ask for rules on how fast anyone may run, the rules rarely inconvenience them. They inconvenience whoever is behind them and cannot afford independent evaluators inside their own building. Others say end-of-the-world stories are convenient precisely because they draw attention away from the harms that already exist, today, with the models as they are.

Amodei closes by saying his desire for the benefits has not faded, but that they will only arrive if the technology is built the right way. That is a sentence hard to argue with - and exactly why it is worth watching who says it and in whose favour the weight of the proposed rules falls.