Databricks executive breaks down the AI talent wars: 'It's like looking for LeBron James'

• Databricks' VP of AI said looking for top researchers is like "looking for LeBron James."
• There are likely less than 1,000 researchers capable of building frontier models, Naveen Rao told Command Line.
• Rao said it's not ridiculous for companies to pay large amounts for AI and infrastructure talent.

Recruiting top-tier AI researchers today is a bit like a sports team scouting for its next star athlete — they're few in number and costly to recruit but can change an organization's trajectory.

"It's like looking for LeBron James," Databricks' VP of AI, Naveen Rao, told The Verge's Command Line newsletter published Friday. "There are just not very many humans who are capable of that."

While thousands of tech workers and engineers are qualified to work on AI, identifying the topmost tiers of AI talent — and convincing them to jump ship — remains a challenge for companies leading the AI race. Rao said he agreed that there are probably fewer than 1,000 researchers capable of building new frontier models. However, the work of a star AI engineer can have a "massive influence" on a company's ability to win, the executive added.

Rao said the AI talent wars aren't just about "pure AI talent," though. They're also about scaling and building infrastructure for AI models. He said he sees some aspects of the pool expanding in that area.

"When you build a model and you want to scale it, that actually is not AI talent, per see," Rao told Command Line. "It's infrastructure talent."

The level of scarcity for top AI talent has given researchers "unprecedented" leverage at the companies they work at, he added. While most Americans are navigating an employer-driven job market, cutting-edge AI engineers seem to have the upper hand.

Earlier this year, Perplexity's CEO shared an instance where he was rebuffed by a Meta engineer who told him to "come back to me when you have 10,000 H100 GPUs," the high-demand Nvidia chips needed to develop and scale AI.

As competition increases, top players in the AI race, like OpenAI, Anthropic, xAI, Microsoft, and Google, have ramped up their hiring efforts. AI tech workers have shared stories of the extravagant efforts CEOs have made to secure top talent. One worker said that OpenAI CEO Sam Altman personally called to pitch them on joining the team. Similarly, a Meta recruit reported Mark Zuckerberg showing up in an email thread.

The companies are also offering hefty pay incentives to secure the best talent. Google recently turned heads when it paid a reported $2.7 billion in a deal to bring 48-year-old Character.ai founder Noam Shazeer back to the company. While Google didn't formally acquire Character.ai, it paid to license the startup's technology, and Shazeer made hundreds of millions from the deal, The Wall Street Journal reported.

Databricks' Rao said while it may sound ridiculous to hear about how much these companies are paying for talent, it's not. The executive cited an example of a former employee at his company Nervana. He described the employee, who now works at OpenAI, as "the best GPU programmer in the world." That programmer's code now likely powers every inference on OpenAI models and could have saved the company $4 billion, Rao said.

"I think that's why you see Google hiring back Noam Shazeer," Rao said. "It's very hard to find another Noam Shazeer."