Artificial Intelligence Nearly Drove the U.S. into Conflict with China

On September 18, CNN revealed that during this year’s springtime tensions between the U.S. and Iran, a military intelligence assessment generated with artificial intelligence nearly prompted U.S. forces to take armed action against a Chinese cargo vessel sailing in the Middle East maritime zone.

According to four individuals familiar with the matter, U.S. forces received an intelligence analysis claiming the Chinese ship was transporting equipment and materials linked to a nuclear weapons program. The report quickly triggered high alert within the U.S. military chain of command, prompting immediate readiness measures for intercepting the target vessel. Two sources confirmed that special operations assault teams had prepared for boarding and inspection; another source and a second individual familiar with the incident stated that military aircraft had already taken off and were on standby—leaving the operation just one step away from execution.

However, before any action could proceed, officials conducted a further verification of the intelligence. They discovered that the assessment had been produced by an analyst at the U.S. Pacific Special Operations Command (based in Hawaii) using an artificial intelligence tool. The analyst had employed a chatbot to process a list of shipping manifests. The AI system, combining open-source information with classified signals intelligence, incorrectly identified the cargo aboard the vessel, leading to the erroneous conclusion that it was involved in a nuclear weapons project. The analyst then used the same AI tool to reformat this flawed conclusion into a document resembling an official summary, which was circulated across multiple command channels.

The actual contents of the ship’s cargo remain unclear, but sources familiar with the situation assert that the intelligence was entirely inaccurate. One individual who reviewed the full sequence described the report as “completely false,” yet emphasized that it “almost sparked a war.”

This incident unfolded amid ongoing efforts by the U.S. military and intelligence community to integrate artificial intelligence across operational domains. From analyzing vast volumes of raw intelligence and identifying targets to decision-making on strikes, budgeting, logistics, and supply chain planning, AI is being progressively embedded into key functions of the military infrastructure.

In January, Defense Secretary Pete Hegseth unveiled a new initiative called the "Artificial Intelligence Acceleration Strategy," aimed at accelerating the adoption of AI within the military by reducing bureaucratic hurdles, increasing funding, and expanding experimental programs—seeking to maintain U.S. leadership in military AI applications.

Previously, Pentagon officials described AI as a transformative technology offering significant advantages in “accelerating the kill chain,” enabling commanders to respond with precision at optimal moments. Yet the very speed that makes AI attractive also increases the risk of hallucinations emerging when human oversight is insufficient, allowing erroneous outputs to propagate unchecked up the command hierarchy.

This episode highlights several deep-seated vulnerabilities in the U.S. military’s use of artificial intelligence:

-- Inconsistent tool standards: It remains unclear whether the chatbot used by the analyst was a commercial product or a government-developed system, underscoring the fragmented deployment of AI tools across military agencies and the absence of unified governance frameworks.

-- Lack of verification mechanisms: The erroneous AI-generated conclusion entered the command chain without sufficient human review, indicating the absence of clear protocols governing human-machine collaboration in target selection processes.

-- Underestimated hallucination risks: Many AI systems currently deployed within military operations are still based on commercial large language models, with no fundamental distinction from widely available chatbots. While capable of rapid information processing, these tools remain susceptible to generating plausible-sounding but factually incorrect conclusions—commonly referred to as “hallucinations.”

Jake Steckler, a researcher at GovAI and a former U.S. Army officer, stated: “For service members, understanding the inherent uncertainty of large language models is critical. But when decisions involving the use of force—such as target selection, intelligence analysis, or operational planning—are at stake, this awareness becomes especially vital. These decisions carry life-or-death consequences.”

This is not the first instance where AI misjudgment has led to military action. On the first day of the U.S.-Iran conflict in February 2026, the U.S. military’s Maven Smart System misidentified a girls’ primary school in Minaab, Iran, as a high-priority military target due to an outdated database. A Tomahawk missile destroyed the building, killing approximately 160 to 175 civilians. Human commanders, influenced by “automation bias,” approved the strike after only about 20 seconds of perfunctory review.

Dario Amodei, CEO of Anthropic, has previously warned: “The technology is not reliable. I am genuinely concerned about fully autonomous weapons—they are not ready to participate in strike decisions.”

The incident has emerged as one of the most illustrative cases of the risks posed by artificial intelligence in military intelligence. According to CNN sources, the error could have triggered a serious escalation in international tensions, potentially leading to armed conflict. Any military action against a Chinese vessel would not only risk immediate confrontation but could also plunge the United States and China into deeper strategic antagonism.

Gary Marcus, a prominent AI expert, commented on social media: “I told the U.S. Senate unequivocally that inaccurate AI-generated information could lead to an accidental war. Now it has happened. The next time, we may not be so fortunate.”

Original: toutiao.com/article/1876716883995660/

Disclaimer: This article reflects the personal views of the author