Anthropic’s forced removal from the U.S. government is threatening critical AI nuclear safety research
The sudden wind-down of Anthropic technology within the U.S. government is raising concerns about whether federal officials, without access to Claude, might fall behind in the quest to guard against the threat of AI-generated or AI-assisted nuclear and chemical weapons.
Though the rollout has been messy—and Claude remains in use in some parts of the government—the Trump administration’s anti-Anthropic posture could have a chilling effect on collaborations between AI companies and federal agencies, including partnerships focused on critical national security questions related to these kinds of futuristic threats, several sources tell Fast Company. The worry is that severing ties with the company could both limit government researchers’ understanding of how, in the future, bad actors could use AI to generate new types of nuclear and biological weapons, and hold back scientific progress more broadly.
Since at least February 2024, Anthropic has participated in a formal partnership with the National Nuclear Security Administration, the federal agency charged with monitoring the country’s nuclear stockpile. The point of that work, the company has previously said, is to “evaluate our AI models for potential nuclear and radiological risks.” The concern, here, is that developing nuclear weapons requires specialized knowledge, but that AI, as it continues to advance, could eventually become adept at developing this expertise on its own. Eventually, a large language model might be able to help someone figure out how to design an incredibly dangerous weapon—or even come up with a novel one itself.
Now, in the wake of President Donald Trump’s Truth Social post demanding that federal workers stop using Anthropic tech, it’s not clear what might happen to Anthropic’s efforts to guard against these future threats. Some federal agencies appear to still be weighing how to approach the Claude use cases they already have, while others are cutting off access to the tool entirely.
“As directed by President Trump, the Department of Energy is reviewing all existing contracts and uses of Anthropic technology,” a spokesperson for the NNSA tells Fast Company. “The Department remains firmly committed to ensuring that the technology we employ serves the public interest, protects America’s energy and national security, and advances our mission.” Anthropic declined to comment.
Safety concerns at the Energy Department
For the past few years, Anthropic has been collaborating with or providing technology to the myriad agencies and national labs that fall under the Department of Energy. For instance, the Lawrence Livermore National Laboratory began using Claude for Enterprise in 2025 and, at the time, made the tool available to about 10,000 scientists. The lab said last year that the technology was supposed to help accelerate its research efforts “in the domains of nuclear deterrence, energy security, materials science,” and other areas.
Anthropic has also worked with the National Nuclear Security Agency on evaluating potential AI-related nuclear safety risks. For example, the agency has provided Anthropic with “high-level” metrics and guidance that have helped the company analyze the threat of its own technology. Anthropic has also worked with the NNSA on developing technology that can scan and categorize AI chatbot conversations and search for signs that someone might be using an LLM to discuss building a nuclear weapon.
An inventory for 2025 for the Department of Energy disclosed that the agency was using Claude at the Pacific Northwest National Laboratory, the Lawrence Livermore National Laboratory, and the Idaho National Laboratory in pilots. Anthropic is also one of several partners in the agency’s Genesis mission, which aims to accelerate scientific development by leveraging artificial intelligence.
Those collaborations may now be in jeopardy. Claude is “everywhere” in the Energy Department’s labs, including at the NNSA, according to Ann Dunkin, the department’s former chief information officer. If labs, or the NNSA, “are working on projects using Anthropic as their AI tool, they are going to have to, at the very least, stop work and start with a new vendor,” Durking tells Fast Company. “This will cost time and money. More than likely, there will be [new] work as they will have to train a new model.” To conduct simulations that involve studying various AI risks, it’s important to understand how all AI models might behave, she adds.
In regard to nuclear weapons, there’s worry that AI could be used to gather enough information to build one such weapon—or be jailbroken so that it could provide that information, Dunkin says.
A former Department of Homeland Security official who focused on AI safety issues echoes those concerns. Anthropic, the person tells Fast Company, was a leader on evaluating how AI models, including its own, might create serious safety risks related to chemical and nuclear weapons. Pressure to remove Anthropic risks wasting peoples’ time and may not be successful anyway, they say. It also puts federal officials behind on trying to understand the full risks related to artificial intelligence, or to fully benefit from its efficiencies, given that Anthropic is still the leading provider of some AI capabilities. “There’s no ban on Claude for the bad guys,” the former official adds.
Overall, the government’s sudden turn against Anthropic risks scaring off other companies that might want to work on serious issues, including those related to nuclear security. “Anthropic learned that once you’re serving the U.S. government, you might not have the right to say no, at least now without retaliation. Naturally that will deter others from working for the government, especially on sensitive topics,” says Steven Adler, an ex-OpenAI employee who focuses on AI safety issues.
“There’s a bitter irony here: The administration is simultaneously demanding AI companies help with national security and making it harder for responsible actors to do exactly that,” Alex Bores, who is running for a House seat in New York on a platform focused on AI regulation, tells Fast Company in a statement. “AI companies working with NNSA to evaluate risk isn’t a liability—it’s a model. Punishing it sends exactly the wrong signal at exactly the wrong time.”
An incomplete exit plan
It’s not immediately clear how federal agencies are supposed to approach Anthropic technology right now. Trump used Truth Social to demand that federal agencies “immediately cease all use of Anthropic’s technology,” though such instructions are ordinarily communicated through the federal chief information officer. The Trump administration is reportedly working on an executive order related to Anthropic, while Anthropic has filed a lawsuit challenging its designation as a “supply chain risk.”
The General Services Administration, according to one post, seems to be interpreting the Truth Social post as a national security directive. The agency’s GitHub repository shows that Claude was recently removed for its interagency AI resource, and a person within the agency confirmed that employees could no longer access Claude internally. Still, another person at the agency tells Fast Company that no official instructions about how to actually enforce removing Claude from federal use cases have actually been sent to employees.
One major challenge with stripping Anthropic’s technology from the federal government is that the technology can be delivered in many ways. In Claude’s case, this includes products sold by Anthropic directly, but also integrations with popular—and controversial—government contractors like Palantir and Amazon Web Services.
Notably, Claude for Government is still listed as one of the features offered within the Palantir Federal Cloud Service, and several agencies have authorized the use of this package, including the Brookhaven National Lab and the Environmental Management Consolidated Business Center, as well as the State Department and the Treasury. The product describes Claude as “purpose built” to meet high government security requirements. Palantir also has a long-standing relationship with the NNSA that predates LLMs.
The NNSA spokesperson declined to comment on how they were approaching the use of Claude and classified systems. At the time of this writing, Palantir had not respondd to a request for comment.
On the military side of government, much ado has been made of the fact that only Caude, and not systems like ChatGPT, has been cleared to operate in classified environments. The Pentagon has since sent a memo to employees that prioritizes removing Claude from any systems that involve nuclear security. Classified environments are also important to civilian agencies. Though Treasury Secretary Scott Bessent has said his agency will be “terminating” use of Anthropic products and Claude, there was at least some grumbling at a recent meeting focused on AI use within the agency that other AI tools weren’t similarly available for classified information.