Vatican AI adviser warns major labs operate like a cartel

Paolo Benanti, a priest who advises the Catholic Church on artificial intelligence, argues that dramatic warnings about superintelligence are diverting attention from the real problem: the concentration of power in a handful of companies that write their own rules. In an article published after a wave of warnings from researchers inside the major labs, he writes that the "frontier labs" narrative serves a clear commercial interest — preserving market position — and when a small group of players dictates regulation for an entire sector, "that has a very clear name: a cartel."

Ethics cannot belong to the boardroom

Benanti says ethics cannot be the preserve of a private company's fiduciaries; it must be balanced and tempered through democratic public debate. He criticizes a call by Anthropic chief executive Dario Amodei for mandatory third-party evaluations before model release and for a crackdown on distillation, the process by which rival labs train models on the output of proprietary ones. Benanti fears restrictions on development methods will choke innovation, and prefers standards that guarantee models do not escape human control — standards not written by the labs themselves.

Overlapping incentives undermine trust

It is hard to know how seriously to take grandiose claims about the capabilities of current and future models, Benanti says, because economic, political and personal incentives are all in play at once. Both OpenAI and Anthropic publicly support slowing development, yet both are preparing for public listings with billions of dollars at stake for whoever captures the market. The rivalry between the United States and China, where the most advanced models are also being built, adds another layer of complexity. Anthropic declined to comment; OpenAI and Google DeepMind did not respond to requests for comment.

Not divinity, statistical computation

Benanti also attacks the language parts of the industry use to discuss superintelligence, with a reverence usually reserved for the divine — as a sophisticated entity beyond human understanding and effectively all-powerful. OpenAI chief executive Sam Altman has previously said he aims to build "magical intelligence in the sky." That framing, Benanti argues, warps the conversation: "Are we talking about something, or someone? That is the real difference. This is not a question of faith, it is a question of mathematics. We are not building the next divinity. We are only deceiving ourselves into thinking there is something beyond statistical computation."

Religion in the new public square

The very effort to define what separates human from machine, and how to protect future generations from the technology's negative consequences, may create "a new role for religions in public discourse," he says. Grappling with the challenge of time is part of the Church's mission. Benanti stresses that he does not officially represent the Holy See, the governing body of the Catholic Church, but advises it in his capacity as a researcher of philosophy, theology and technology.