AI nearly pulled the United States into a military operation against China

U.S. fighter jets were already airborne, heading for a Chinese vessel last spring, when intelligence driving the armed sortie turned out to be a chatbot hallucination. The operation was called off at the last minute, averting a possible clash with Beijing, CNN reported Friday. The episode shows how fast a language-model error can climb the chain of command without anyone stopping to ask questions.

The intelligence report, which circulated at the highest levels during the conflict with Iran, claimed the ship was carrying components for a military nuclear program. The error originated with an analyst at Special Operations Command who asked a language model to process open-source intelligence alongside classified signals. The model misread the cargo manifest, and the analyst re-ran it to shape the faulty findings into an official-looking summary that spread through command channels.

The Pentagon is pushing accelerated AI adoption to shorten the "kill chain" and keep an edge over China. That same speed lets hallucinations slip through without adequate human oversight. When the system produces output that looks convincing, the ease with which it is adopted as legitimate intelligence product becomes a risk multiplier.

"Life and death depend on this," Jake Steckler, a researcher at GovAI and former U.S. military officer, told TechCrunch. Service members must understand the inherent uncertainty in large language models, especially when decisions involve use of force, targeting, intelligence analysis, or operational planning. The incident should be a wake-up call for adding safeguards, not a reason to abandon the tools; prioritizing adoption speed over safety will only erode trust and slow adoption in the long run.