The AI risk that few organizations are governing

Most enterprises can tell you how many human users have access to their financial systems. Few can tell you how many AI agents do. 

In recent years, enterprise AI discussions have centered on workforce disruption, return on investment and the mechanics of scaling use cases. Those questions, while important, are increasingly operational. A more structural issue is emerging, one that will define whether AI becomes a durable advantage or a compounding liability.

The real risk is not model performance or media hype. It is the rapid proliferation of autonomous AI agents operating without governed identity, enforceable access controls or lifecycle governance. Governance frameworks designed for human users and traditional software are being quietly outpaced – and few organizations are systematically measuring the exposure.

Recently, this issue has become more visible, with platforms emerging that have no real safeguards to prevent bad actors and the capacity to create and launch huge fleets of bots. These platforms illustrate how quickly unmanaged digital actors can proliferate – and how difficult they become to track once they do. Intelligent programs are now working without meaningful governance and access to systems and data beyond our visibility. 

If organizations don’t implement industrial-grade security frameworks for AI agents today, we will quickly face the consequences in mission-critical enterprise environments.

Unchecked AI agents: The next enterprise risk frontier

AI agents differ in important ways from both traditional software and human users. Most enterprise systems today are built around clearly defined identities. Users have named accounts, applications operate with registered service credentials and access is granted according to established roles that can be monitored, audited and revoked when necessary.

Autonomous AI agents do not fit neatly into this model. They can act on behalf of users, interact with multiple systems and make decisions without direct human intervention. In many organizations, they lack stable, governed identities. Their access is not always tied to clear policies. Their lifecycle is rarely managed from creation through retirement.

Researchers have highlighted how weaknesses in agent-driven environments can allow malicious instructions, prompt injection attacks or poisoned data to propagate rapidly across interconnected systems. In enterprises where agents are connected to sensitive data, financial systems or operational infrastructure, even small governance gaps can escalate into material risk.

In other words, the real risk isn’t just what the agents can do, it’s what they can access. 

The real vulnerability isn’t the AI model, it’s the foundation

In my work with organizations moving from AI experimentation to enterprise-scale deployment, one pattern stands out: the biggest points of failure are rarely the AI models themselves. More often, the issue is weak data foundations and incomplete control frameworks. 

The consequences are already tangible. Compliance failures, biased outputs and governance breakdowns are generating material financial and operational losses across industries. In several cases, remediation costs have escalated into the tens of millions when governance gaps are discovered post-deployment. These are not examples of runaway intelligence. They are operational failures. When AI is introduced into complex environments without modernized identity governance and continuous monitoring, risk scales faster than value.

The urgency intensifies as AI adoption spreads beyond centralized teams. Employees are experimenting with and deploying agents inside business functions, often without enterprise-wide visibility. Autonomy is expanding laterally across organizations faster than enterprise oversite can adapt. Without clear standards for identity, access and oversight, digital actors can quietly accumulate permissions and influence well beyond their intended scope.

This is ultimately a question of architectural readiness. Leadership should be able to answer three questions at any time: Where does our critical data reside? Who or what can access it? How is that access validated and reviewed?  

Scaling AI safely therefore requires an operational reset. Autonomous agents must be treated as accountable actors within the enterprise. This includes clear documentation of roles and responsibilities, regular review cycles and integration with existing IT and risk processes. Access should be intentional and continuously validated and activity must remain observable. Organizations that make this shift are not constraining innovation; they are creating the conditions for sustainable scale. In the AI era, operational maturity is what ultimately separates experimentation from durable advantage.

A call to shift the narrative from hype to preparedness

AI agents aren’t a theoretical threat anymore and it’s clear that the broader industry conversation needs to evolve. We spend a great deal of time discussing model performance and new use cases. We need to spend just as much time on identity, data governance, access control and lifecycle management for the autonomous actors we are introducing into our environments.

Without the guardrails long standard in other areas of IT, these agents can represent a quiet army of unmanaged digital actors operating inside complex systems. Addressing that risk requires leadership attention, cross-functional collaboration and a commitment to building industrial-grade governance for the AI era. Organizations that take this seriously will not only reduce their exposure. They will also build the trust and resilience needed to scale AI with confidence, fostering stronger collaboration between business and IT. In a world where intelligent systems are becoming part of the workforce, operational security is no longer just a technical concern, but a strategic imperative. AI will scale only as far as trust allows it to. Governance is what makes that trust possible.

The views reflected in this article are the views of the author and do not necessarily reflect the views of the global EY organization or its member firms, nor do they necessarily reflect the opinions and beliefs of Fortune.

This story was originally featured on Fortune.com