Companies are deploying AI agents fast. They still need to make them scale.

• AI agents can take on some tasks, yet challenges can arise when companies link them together.
• One longtime tech exec says that's why, for now, autonomous AI remains aspirational.
• Both leaders and workers will need to rethink job roles as AI evolves, tech execs told Business Insider.

Despite some CEOs' enthusiasm, AI agents aren't always up to the task.

One culprit: hallucinations.

Agents are software programs that can perform tasks independently, such as reconciling an invoice, so that a person doesn't have to. Sounds great, right? That's why companies are plugging agents into tasks across sales, finance, supply chains, and engineering.

Yet while agentic systems can be effective at completing discrete tasks, they can be fragile at scale.

Errors can quickly become a bottleneck if hallucinations multiply when agents interact, said Nicolas Darveau-Garneau, a former Google executive and author of the book "Be a Sequoia, Not a Bonsai."

If a single agent has a 5% hallucination rate, then it's hard to daisy-chain multiple agents without a high risk of errors. That's because the risk increases exponentially, he told Business Insider.

"The only way we get massive productivity gains is the day that we can trust that the AI is not hallucinating a lot," Darveau-Garneau said.

For now, he said, that's why general-purpose, autonomous agentic AI is largely aspirational.

Nevertheless, hallucinations are something that Darveau-Garneau expects will be "mostly solved" five years from now.

If that happens, he said, the productivity gains of 10% to 30% that some companies are getting from AI could be far greater — perhaps on the order of 10x to 100x.

Those gains matter because the bigger they get, the more likely it becomes that companies will reorganize their operations, including the possibility of needing fewer employees.

Even if companies don't cut staff, many roles — especially desk jobs — are likely to change.

Scaling will require rethinking how companies are structured

The challenge of scaling agents and AI broadly is one reason Darveau-Garneau expects companies to still need some humans in jobs that sometimes appear to be endangered, like coding.

An excellent software engineer, he said, might be 30% to 40% more productive with AI. It's a nice lift, Darveau-Garneau said, but not a reason for a company to get rid of all its engineers.

Workers are understandably fearful of losing their jobs to AI, especially when leaders emphasize productivity gains, he said.

Darveau-Garneau said that the CEO's primary job when it comes to AI is to create the right environment for adoption — including training and making sure that employees feel "psychologically safe."

That's why, he said, leaders would be wise to reframe the conversation to say they want to double a company's size, for example, while increasing the employee base by 30%, rather than suggesting they deploy AI and cut 20% of the staff.

Execs know change is coming

One challenge for many leaders has been understanding what new roles will look like thanks to AI, Muqsit Ashraf, group chief executive of strategy at Accenture, told Business Insider.

That is proving difficult, at least in part, because some current job descriptions may change, and other jobs may become redundant. Still others will have to be "radically redefined," he said.

"What companies have been doing, by and large, is putting AI in the hands of people in their existing roles with existing tasks," Ashraf said.

That might mean automating some tasks or using AI to supercharge how someone searches for information or how quickly they can conduct a competitive analysis.

"But how does that fundamentally change my day?" he said.

Companies have struggled, Ashraf said, if they've tended to think about adopting AI within their current ways of working, rather than redesigning the organization with an eye toward future jobs.

"It's not about layering AI on existing roles. It's about rethinking all of these processes and activities," he said.

Despite the challenge of redesigning how companies are set up, Ashraf said it's encouraging that many C-suite leaders recognize the need to help their workers build the skills needed to prepare for the changes stemming from companies' decisions to prioritize AI.

AI might still have limitations, but its rapid pace of improvement means workers themselves also need to prioritize staying current. Otherwise, they risk falling behind, Alex Salazar, cofounder and CEO of the AI infrastructure startup Arcade, told Business Insider.

A year ago, the coding agents that the company's software engineers now use couldn't do any of what they can do today, he said.

"What does that mean for your job as an engineer?" Salazar said. "It's got to evolve."

Do you have a story to share about your career? Contact this reporter at tparadis@businessinsider.com