Want AI agents to work better? Improve the way they retrieve information, Databricks says

Hello and welcome to Eye on AI. In this edition…Nvidia snags the team and tech from AI chip startup Groq…Meta buys Manus AI…AI gets better at improving AI…but we might not know enough about the brain to reach AGI.

Happy New Year! A lot has happened in AI since we signed off for the year just before Christmas Eve. We’ll aim to catch you up in the Eye on AI News section below.

Meanwhile, as I’ve noted here before, 2025 was supposed to be the year of AI agents, but most companies struggled to implement them. As the year drew to a close, most companies were stuck in the pilot phase of experimenting with AI agents. I think that’s going to change this year, and one reason is that tech vendors are figuring out that simply offering AI models with agentic capabilities is not enough. They have to help their customers engineer the entire work flow around the AI agent—either directly, through forward deployed engineers who act as consultants and “customer success” sherpas; or through software solutions that make it super easy for customers to do this work on their own.

A key step in getting these workflows right is making sure AI agents have access to the right information. Since 2023, the standard way to do this has been with some kind of RAG, or retrieval augmented generation, process. Essentially, the idea is that the AI system has access to some kind of search engine that allows it to retrieve the most relevant documents or data from either internal corporate sources or the public internet and then the AI model bases its response or takes action based on that data, rather than relying on anything it learned during its training process. There are many different search tools that can be used for a RAG system—and many companies use a hybrid approach that combines vector databases, particularly for unstructured documents, as well as more traditional keyword search or even old-fashioned Boolean search.

But RAG is not a panacea and simple RAG AI processes can still suffer from relatively high error rates. One problem is that AI models often struggle to translate a user’s prompt into good search criteria. Another is that even if the search is conducted well, often the model fails to properly filter and sift the data from an initial search. This is sometimes because there are too many different data formats being retrieved, and sometimes because the human who is prompting the AI model has not written good instructions. In some cases, the AI models themselves are not reliable enough and they ignore some of the instructions.

But, most of the time, AI agents fail not because the agent “is not able to reason about data but the agent is not getting the right data in the first place,” Michael Bendersky, the research director at Databricks tells me. Bendersky was a long-time veteran of Google, where he worked on both Google Search and for Google DeepMind.

Databricks introduces a new retrieval ‘architecture’ that beats RAG

Today, Databricks (known for its data analytics software) is debuting a new architecture for retrieval-augmented AI agents called Instructed Retriever that it says solves most of RAG’s shortcomings.

The system translates a user’s prompt and any custom specifications that the model should always consider (such as the recency of a document or whether a product has good customer reviews) into a multi-step search plan for both structured and unstructured data—and, crucially, metadata—to get the right information to the AI model.

Much of this has to do with translating the natural language of the user’s prompt and the search specifications into specialized search query language. “The magic is in how you translate the natural language, and sometimes it is very difficult, and create a really good model to do the query translation,” Hanlin Tang, Databricks’ CTO for neural networks, says. (Tang was one of the cofounders of MosaicML, which Databricks acquired in 2023.)

On a suite of benchmark tests that Databricks designed that it says reflects real world enterprise use cases involving instruction-following, domain-specific search, report generation, list generation, and searching PDFs with complex layouts, the company’s Instructed Retriever architecture resulted in 70% better accuracy than a simple RAG method and, when used in a multi-step agentic process, delivered a 30% improvement over the same process built on RAG, while requiring 8% fewer steps on average to get to a result.

Improving results even with under-specified instructions

The company also created a new test to see how well the model can deal with queries that may not be well-specified. It is based partly on an existing benchmark dataset from Stanford University called StaRK (Semi-structured Retrieval Benchmark). In this case, Databricks looked at a subset of these queries related to Amazon product searches, called StaRK-Amazon, and then further augmented this dataset with additional examples. They wanted to look at search queries that have implied conditions. For instance, the query, “find a jacket from FooBrand that is best rated for cold weather,” has multiple implied constraints. It has to be a jacket. It has to be from FoodBrand. It has to be the FooBrand jacket that has the highest rating for cold weather. They also looked at queries where users want to exclude certain products or want the AI agent only to find products with recent reviews.

The idea of the Instructed Retriever architecture is that it turns these implied conditions into explicit search parameters. Bendersky says the breakthrough here is that Instructed Retriever knows how to turn a natural language query into one that will leverage meta data.

Databricks tested the Instructed Retriever architecture using OpenAI’s GPT-5 Nano and GPT-5.2, as well as Anthropic’s Claude-4.5 Sonnet AI models, and then also a fine-tuned small 4 billion parameter model they created specifically to handle these kind of queries, which they call InstructedRetriever-4B. They evaluated all of these against a traditional RAG architecture. Here they scored between 35% to 50% better in terms of the accuracy of the results. And the Instructed Retriever-4B scored about on par with the larger frontier models from OpenAI and Anthropic, while being cheaper to deploy.

As always with AI, having your data in the right place and formatted in the right way is the crucial first step to success. Bendersky says that Instructed Retriever should work well as long as  an enterprise’s dataset has a search index that includes metadata. (Databricks also offers products to help take completely unstructured datasets and produce this meta data.)

The company says that Instructed Retriever is available today to its beta test customers using its Knowledge Assistant product in its Agent Bricks AI agent building platform and should be in wide release soon.

This is just one example of the kinds of innovations we are almost certainly going to see more of this year from all the AI agent vendors. They might just make 2026 be the real year of AI agents.

With that, here’s more AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

This story was originally featured on Fortune.com