New US policy on AI threatens industry disruption, puts US at loggerheads with Holy See

AI giant Anthropic is facing potentially disastrous backlash from U.S. Secretary of Defense Pete Hegseth and the recently renamed War Dept. over Anthropic’s refusal to allow the Pentagon to use their Claude AI platform in the design or deployment of autonomous AI weapons systems.

Colloquially known as “killer robots” and officially styled LAWS – Lethal Autonomous Weapons Systems – the very idea of such systems has been controversial for generations, and the Holy See has been voicing opposition to the development of autonomous weapons systems for more than a decade in various international forums.

What was once a universe-building element in dystopian and apocalyptic science fiction, however, is now reportedly driving Pentagon policy.

The Pentagon’s sour turn on Anthropic – a San Francisco-based public benefit corporation dedicated to the safe and ethical development and use of AI – is the result of the company’s refusal to see its tech used to spy on U.S. citizens en masse or to develop weapons systems designed to operate without any human involvement.

For that reason, the Pentagon is reportedly “close” to severing all ties with Anthropic and is also seriously considering a “supply chain risk” designation for the company, which would mean that any company wishing to do business with the U.S. military would have to sever any existing ties with Anthropic and avoid doing business with the company.

Contours of a crisis

In late January, Anthropic published a new “Constitution” for the company’s flagship Large Language Model, Claude, organized around principles the company says are designed to keep Claude from “undermining appropriate human mechanisms to oversee AI during the current phase of development,” among other safeguards and guidelines.

The publication of Anthropic’s Claude principles for the ethical development and use of Claude comes as the company emerges from legal difficulties. A major class action lawsuit saw Anthropic enter a $1.5 billion settlement agreement over the unauthorized use of copyrighted material to train earlier iterations of its model.

“Supply chain risk”

The Pentagon’s “supply chain risk” designation is usually reserved for foreign adversaries suspected of posing a security risk to the United States, but Anthropic’s Claude AI is already integrated into many U.S. military systems.

“It will be an enormous pain in the ass to disentangle,” Axios quoted a senior Pentagon official as saying on Feb. 16, “and we are going to make sure [Anthropic] pay[s] a price for forcing our hand like this.”

Chief Pentagon spokesman Sean Parnell confirmed to Crux that the Department’s relationship with Anthropic is under review.

“The Department of War’s relationship with Anthropic is being reviewed,” Parnell told Crux.

“Our nation requires that our partners be willing to help our warfighters win in any fight,” he said in response to email queries. “Ultimately, this is about our troops and the safety of the American people,” Parnell also said.

That is in keeping with statements Hegseth made earlier this year when he unveiled the Pentagon’s Artificial Intelligence Acceleration Strategy.

“We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI,” Hegseth said on Jan. 12.

In a Jan. 9 memorandum detailing the new policy – under a heading titled “Clarifying ‘Responsible Al’ at the DoW – Out with Utopian Idealism, In with Hard-Nosed Realism – Hegseth wrote: “[W]e must not employ AI models which incorporate ideological ‘tuning’ that interferes with their ability to provide objectively truthful responses to user prompts.”

That was a reference to “Diversity, Equity, and Inclusion and social ideology,” which “have no place in the DoW [sic].”

“The Department,” Hegseth wrote, “must also utilize models free from usage policy constraints that may limit lawful military applications.”

Hegseth directed the Department’s Chief Digital and Artificial Intelligence Office “to establish benchmarks for model objectivity as a primary procurement criterion within 90 days,” and directed the Department “incorporate standard ‘any lawful use’ language into any DoW contract through which AI services are procured within 180 days.”

Just exactly what is lawful use, however, is an outstanding question for contractors, lawmakers, military officers, and even DoD/DoW officials.

Industry disruption

Whatever the reason for the move, observers say blacklisting Anthropic could cause serious disruption across the AI industry, not least for Amazon and OpenAI, both of which have invested billions of dollars in Anthropic and rely heavily on Anthropic’s cloud systems.

If the Pentagon does blacklist Anthropic, any company currently doing business with the outfit could be faced with a stark choice: To cut ties with Anthropic and do business with the U.S. military, or else keep doing business with Anthropic and forego business with the Pentagon.

Hegseth’s January memorandum said the department would “measure success through continuous field experimentation: putting AI capabilities in operators’ hands, gathering feedback within days not years, and pushing updates faster than the enemy can adapt.”

A major legal and ethical concern, however, is that autonomous weapons systems will operate – by design – without human direction or control.

“Together with capability innovation,” Hegseth stated in his memo, “we must more fully incorporate AI and Autonomy into military planning; tactics, techniques and procedures (TTP) development; and experimentation processes.”

U.S. policy collides with Holy See

This appears to put official U.S. defense policy as articulated by the Secretary – who is Catholic – at loggerheads with the Holy See.

Queries from Crux to the press office of the Holy See were without reply at time of publication, but an August 2024 statement from the Permanent Observer to the UN and other international offices in Geneva to a high-level meeting reiterated the Holy See’s desire to see a legally binding agreement to regulate research into the weaponization of AI and a moratorium on the development and use of such weapons.

In January of this year, Pope Leo XIV highlighted the danger of AI weapons systems in his first “state of the world” remarks to the diplomatic corps.

“[T]here is a danger of returning to the race of producing ever more sophisticated new weapons, also by means of artificial intelligence,” Leo said. “The latter,” AI, “is a tool that requires appropriate and ethical management, together with regulatory frameworks focused on the protection of freedom and human responsibility,” he said.

This is not an occasional or transient concern for the new pontiff, who took his regnal name, Leo, as a nod to the first Leo and to Leo XIII, two of his predecessors who reigned during times of immense social, civilizational, and spiritual disruption.

Leo I – Leo the Great – reigned in the waning days of Roman imperial power in the West and famously outfaced Attila the Hun. Leo XIII gave the Church encyclical letter, Rerum novarum, the founding document of the Church’s social doctrine in the modern era.

The seminal 1891 encyclical “addressed the social question in the context of the first great industrial revolution,” as Leo XIV explained in remarks to the college of cardinals assembled in the Vatican’s New Synod Hall on May 10, just two days after his election.

“In our own day,” Leo said in those May 10 remarks, “the Church offers to everyone the treasury of her social teaching in response to another industrial revolution and to developments in the field of artificial intelligence that pose new challenges for the defense of human dignity, justice and labor.”

Concerns across recent pontificates

The Vatican began tackling these challenges several years ago, and sponsored a series of conferences and a Call for AI Ethics during the pontificate of Francis.

Originally signed in February 2020 by the Pontifical Academy for Life, Microsoft, IBM, the UN’s Food and Agriculture Organization, and Italy’s Ministry of Innovation, the Rome Call for AI Ethics is an ongoing initiative promoting an ethical approach to AI. The Rome Call involves leaders from tech and industry, government, academia, and civil society.

RELATED: Tech heads join Vatican in ‘clarion call for a new generation of ethics’

In a message to participants in the Second Annual Rome Conference on Artificial Intelligence, which took place June 19-20 of last year, Leo said, “[T]he Church wishes to contribute to a serene and informed discussion,” of pressing questions surrounding the development and use of AI, “by stressing above all the need to weigh the ramifications of AI in light of the ‘integral development of the human person and society’.”

“This entails taking into account the well-being of the human person not only materially, but also intellectually and spiritually,” he said, “it means safeguarding the inviolable dignity of each human person and respecting the cultural and spiritual riches and diversity of the world’s peoples.”

“Ultimately,” Leo told participants including an Anthropic delegation, “the benefits or risks of AI must be evaluated precisely according to this superior ethical criterion.”

Follow Chris Altieri on X: @craltieri