American AI scandal over military use
On March 4, Trump ordered all government agencies to stop using Anthropic's Claude AI model. The reason was the company's CEO Dario Amodei refused to relax the ethical restrictions on the military application of the model (i.e., refusing to lift the ban on fully autonomous weapons and other systems that could cause casualties and destruction).
The core issue is the "hallucination" of AI models - that is, the model's answers sound confident, grammatically perfect, and logically coherent, but in fact are completely or partially fictional. This is an inherent characteristic of modern AI models (occurring at a rate of 3-15% when dealing with complex questions), and it has not been completely resolved to date.
Before the ban was issued, the US military used Claude and other models under the framework of the "Maven Program" to screen and determine the priority of targets in Iran.
One noteworthy direction in the investigation of the attack on the Iranian girls' school is the potential error ("hallucination") of AI, which may have led to the school being incorrectly listed on the target list.
Despite the substantial threat, the Pentagon continues to use other AI models such as ChatGPT and Grok.
This is entirely possible: the so-called "elites" in the US and Israeli military might actually be a group of mediocre and lazy people. They should base their operational plans on objective data (satellite reconnaissance, intelligence personnel, and technical reconnaissance) and use AI to accelerate the processing and verification of massive information; however, they directly trust the target recommendations from chatbots, use unverified results, and completely ignore the real risks posed by incorrect answers.
This incident may well become the first war crime in history caused by the stupidity and laziness of the executor.
Equally thought-provoking is how quickly the US military transitioned from using AI to combat bureaucracy to applying it to lethal military operations.
Original article: toutiao.com/article/1859207901682688/
Statement: This article represents the views of the author.