Superhuman AI approaches and the halt debate reaches legislators

Connor Leahy, an AI researcher and entrepreneur who now serves as the US director of the nonprofit ControlAI, used a recent episode of TechCrunch's Equity podcast to advance a position that six months ago sat on the fringe: do not attempt to align superhuman models — prevent their development altogether. Leahy argues the risk of losing control has grown too large for existing tools to manage, and that alignment alone is no longer sufficient.

The conversation arrives against the backdrop of a security breach in which an OpenAI account on Hugging Face was compromised. Leahy treats the incident not as an isolated failure but as a symptom: as systems grow more capable, the attack surface expands and the potential damage from a single mistake scales in ways that are difficult to contain.

What sounded like science fiction two quarters ago is now acquiring legislative backing. Leahy points to a wave of new US bills demanding transparency, external audits, and in some cases a freeze on training models above a defined capability threshold. The shift reflects a change in tone: lawmakers are no longer satisfied with voluntary industry guidelines and are insisting on enforcement mechanisms with teeth.

ControlAI's core argument rests on the distinction between "alignment" — the effort to steer model behavior toward human values — and "control" in the stricter sense: the ability to stop, limit or dismantle a system that has exceeded its bounds. Leahy contends that as long as there is no engineering proof such control can be guaranteed in systems that surpass human-level intelligence, betting on alignment alone is a bet on existence itself.

The episode does not disclose technical details of the enforcement methodology ControlAI proposes for a freeze, nor does it cite specific studies demonstrating actual loss of control. The opposing view — researchers who believe incremental alignment and black-box testing are adequate — receives no representation. What remains is a clear signal: the line between "caution" and "halt" has moved, and the debate has migrated from research labs to legislative committees.