Anthropic was the first AI company to have its tools used by the military. But the tech giant is refusing calls from the Pentagon to lift restrictions on the use of its powerful Claude AI model. 

On Tuesday, Defense Secretary Pete Hegseth met with Anthropic CEO Dario Amodei to deliver the company an ultimatum: either grant the military unlimited access to its AI model, allowing use for “all lawful purposes,” or lose its $200 million contract and be labeled a “supply chain risk.” This classification is typically reserved for companies tied to U.S. adversaries and could ban companies that work with the government from partnering with Anthropic. 

At the same time, Hegseth threatened to invoke the Defense Production Act, a Cold War-era law that authorizes the government to force the actions of private companies to meet the needs of national defense. This act was used as recently as 2021 to accelerate the production of COVID-19 vaccines, but experts say that its use in the situation with Anthropic would be unprecedented. 

AI policymakers, as well as Amodei himself, have criticized the contradictory nature of the Pentagon’s two threats. According to Amodei, “one labels us a security risk; the other labels Claude as essential to national security.” Dean Ball, a former AI advisor in the Trump administration, said that it was “incoherent” to even float the two policy ideas together, and “a whole different level of insane to move up and say we’re going to do both of those things.” 

Although Amodei indicated that Anthropic would be willing to make some adjustments, he said that the company “cannot in good conscience accede” to the demands of the Pentagon. Anthropic has held fast on two ethical red lines: the use of its model for mass surveillance of U.S. citizens and the development of fully autonomous weapons. 

In a post uploaded to Anthropic’s website, Amodei stated, “mass domestic surveillance is incompatible with democratic values. AI-driven mass surveillance presents serious, novel risks to our fundamental liberties.” The current legality of such surveillance is only “because the law has not yet caught up with the rapidly growing capabilities of AI.” 

The threat to privacy that surveillance systems present has garnered more attention recently. The doorbell camera company Ring faced intense backlash after running a Super Bowl ad demonstrating the company’s “Search Party” function: a lost pet’s owner can alert Ring, which in turn uses AI to search through video from nearby cameras to search for matches. 

Fully autonomous weapons, powered by AI, are able to search, identify, and engage targets without human intervention. These weapons raise a number of significant concerns. From an ethical standpoint, critics argue that it is morally wrong to leave life-or-death decisions to a machine. They are also concerned about the ability of these machines to distinguish between combatants and civilians.

Furthermore, the use of fully autonomous weapons provides space for an accountability gap: if AI causes a wrongful death, who gets held legally responsible? It would be difficult to blame a commander for the actions of an algorithm, but it would not be any easier to attach fault to the coder.  

In his post, Amodei claimed that “frontier AI systems are simply not reliable enough to power fully autonomous weapons.” The founder framed the conflict in terms of the endangerment of American lives: “We will not knowingly provide a product that puts America’s warfighters and civilians at risk.” 

Regarding fully autonomous weapons, it appears that those outside the U.S. would face the greatest danger. Several current global conflicts would be immediately exacerbated by the use of killing machines. With the largest military in the world and the greatest AI capabilities, the U.S. represents a unique threat to world peace – especially considering Hegseth’s advocacy for aggressive foreign policy. According to Hegseth, the military doesn’t “fight with stupid rules of engagement. We untie the hands of our war fighters to intimidate, demoralize, hunt, and kill the enemies of our country.” 

Anthropic’s decision will set the tone for the actions of other AI companies. But whether it cedes to the Pentagon or not, one thing is clear: we are entering a new age of conflict.