'Too Dangerous to Release' Is Becoming AI's New Normal

On April 16, OpenAI announced GPT-Rosalind, a new AI model targeted at the life sciences. It significantly outperforms their current publicly available models in chemistry and biology tasks, as well as experimental design. As with Anthropic’s Claude Mythos and OpenAI’s GPT-5.4-Cyber, also released this month, the model is not available to the general public—reserved, at least initially, for “qualified customers” through a “trusted access program.” 

The releases signal a new and concerning trend of AI companies deeming their most capable models too powerful to entrust to the general public. “I think frontier developers are restricting access to their most capable models because they are genuinely worried about some of the capabilities these models have,” says Peter Wildeford, head of policy at the AI Policy Network, an advocacy group. 

It is unclear why OpenAI decided to restrict access to GPT-Rosalind in particular. An OpenAI spokesperson said in an email that giving access to trusted partners allows the company to “make more capable systems available sooner to verified users, while still managing risk thoughtfully.”

Who decides? 

The rapid advance of AI capabilities raises the question of whether private companies should be making the increasingly weighty decisions about whether and how potentially dangerous AI models should be built, and who should be allowed to use them. “I think the federal government has a role to play,” says Rep. Mark DeSaulnier, a California Democrat. 

Anthropic’s Mythos release appears to have improved its previously fraught relationship with the White House, which said last week that it had held a “productive and constructive” meeting with Anthropic CEO Dario Amodei. The NSA has also reportedly begun using Claude Mythos. In February, President Trump ordered federal agencies to stop working with the “radical left, woke company,” after a contract dispute with the Pentagon. 

The restrictions on access to the recent series of models were voluntary on the part of Anthropic and OpenAI. But as the risks posed by AI models become more severe and complex, some are calling for stricter external oversight. 

“We don't allow companies to decide how much toxic pollutant they're allowed to put in my child's drinking water—this is the government's decision,” says Connor Leahy, U.S. director of ControlAI, an AI regulation advocacy group. “We can argue [whether] the government is doing a bad job or a good job, but it's about the separation of powers.”

‘Science research and making a bioweapon look very similar’

Dual-use capabilities, such as biological and cybersecurity research, pose a challenge to AI companies. The same tools that help a cybersecurity researcher find and patch vulnerabilities in software can assist a would-be attacker. An AI that helps study viruses could, hypothetically, help a bioterrorist design a more lethal strain. “Cyber defense and cyber offense look very similar,” says Wildeford. “Science research and making a bioweapon look very similar.”

In the past, companies have chosen to restrict these capabilities for everybody. Many chatbots refuse queries on which COVID mutations cause the virus to become more transmissible, for example. While this doesn’t bother the average user, it is a challenge for researchers. “It’s frustrating,” says James Diggans, vice president of policy and biosecurity at Twist Bioscience, a DNA synthesis company. “But I think it’s the right thing to do.”

The recent model releases relax some of these constraints for trusted parties. OpenAI says it grants access to GPT-Rosalind only to organizations with “strong internal controls” that ensure the model will not be misused. Anthropic has partnered with U.S. government agencies and private companies that use Mythos to find and patch cybersecurity vulnerabilities. However, Batalis says that defining “legitimate” researchers is harder outside U.S. institutions, raising equity concerns for international researchers. 

Deciding which models should have restricted access is a delicate balancing act that varies by domain. It’s easy to measure whether an AI model poses a potential cyber threat, says Diggans: “Can they crack existing systems?” Biological research is a more complex, multi-stage process that takes longer than cyberattacks; it’s less clear whether harm would come from a model such as GPT-Rosalind if it were publicly released. “We know that people want to, and do, commit cyber attacks,” Batalis says. “We just don't have that same sample size with the biological risks.” Other domains may become more contentious as AI capabilities continue to advance. Communications campaigns could be seen as propaganda operations in the wrong hands. 

‘Cyber capabilities are going to diffuse’

Open-source models, which can be downloaded and run for free, may change the calculus around AI model restrictions. The capabilities of open-source models have historically lagged proprietary models by three to seven months, according to Epoch AI, a research institute that studies AI progress. This means that, if the trend continues, an AI model with GPT-Rosalind- and Mythos-level capabilities could be publicly available by the end of the year.  “Cyber capabilities are going to diffuse,” the OpenAI spokesperson said. “Defenders need better tools earlier, not later.”

Open-source models could benefit international cyberattackers. In November, Anthropic announced that it had disrupted a Chinese state-sponsored group that was using the company’s paywalled models, by blocking their access to the AI. If similarly capable models are freely available on the open web, this would reduce Western companies’ leverage.

However, some open source developers have previously relied on outputs from leading proprietary systems to help train their models. The recent access restrictions on recent models may slow or stop the diffusion of the most advanced capabilities into open-source models—as long as companies can enforce the restrictions. (Some unauthorized users have reportedly already succeeded in accessing Claude Mythos.)

Whether or not open-source models catch up with the leading proprietary models, Mythos and GPT-Rosalind are the new floor of frontier AI capability—and the growing opportunities and risks posed by AI show no sign of slowing. “I think the government has a strong interest in managing that,” says Wildeford. “I don't really know how you get around the need for government intervention.”