The Super Bowl was made for TV and instant replay was made for visual AI. Here’s how it could be better and what it would look like

With Super Bowl LIX quickly approaching, we’ve witnessed 60 years of technological advances on the football field and probably haven’t realized it. And the best is yet to come because of visual AI. 

Most of us will watch the big game from our living rooms rather than shelling out thousands of dollars for a seat in Levi’s stadium. In exchange for not experiencing the thrill and excitement of watching a football game in person, those of us at home will get the benefit of experiencing the game with a few technological enhancements. Over the past few decades, visual AI has changed how viewers watch the game and how it’s officiated on the field. 

The transformation has been remarkable and we’re honestly only at the beginning of what’s possible with visual AI. How fans experience the game and how referees decipher controversial calls will only get better as visual AI advances. 

So, what is visual AI? Visual AI, also commonly known as computer vision in technical circles, focuses on extracting useful information from images and video. It detects objects, tracks motion, and measures positions. Visual AI builds on that foundation but goes further by incorporating context, reasoning, prediction, and decision support. Rather than simply answering what happened in a frame of video, visual AI asks what that information means, how confident we are, and what action should follow.

We’ve already witnessed early applications of this technology on the football field. Remember when the virtual first-down line first appeared on your television screen? It’s so natural now that you may not remember, but that was one of the first mainstream uses of visual AI in sports broadcasting, and it fundamentally changed how fans watch football games. With visual AI progressing, advances like these are likely going to make the football-watching experience more interactive, engaging, and above all, entertaining. 

Seeing the Whole Field at Once

Today, we’re seeing even more sophisticated applications across professional sports. Tennis virtually eliminated controversial line calls with Hawk-Eye technology. Major League Baseball will officially roll out its Automated Ball-Strike Challenge System in 2026. 

Football presents a particularly compelling opportunity for visual AI solutions. Unlike tennis or baseball, where the action focuses on a single point or zone, football is played across multiple planes simultaneously. Officials must make split-second judgments without access to all the evidence they need. They struggle to simultaneously watch the line of scrimmage, monitor for holding in the backfield, and track whether a receiver’s feet stayed inbounds.

The recent AFC Championship game between Denver and New England illustrated this challenge perfectly. With the score tied 7-7 midway through the third quarter, the Patriots faced a critical 4th-and-1 at the Denver 8-yard line. Quarterback Drake Maye executed a “tush push” to gain what referees ruled a first down, leading to the game’s final points. 

However, video analysis afterward showed significant uncertainty about whether he actually made the line to gain a new set of downs. Visual AI could have synthesized multiple camera angles in real-time, created a precise 3D reconstruction of ball placement relative to the first-down marker, and provided officials with definitive evidence within seconds rather than minutes.

As the game progressed and a snowstorm descended on Denver in the fourth quarter, the field became obscured by a blanket of white. These are exactly the conditions where visual AI excels. Given the precalibrated geometry of the field, it can cut through weather interference, track the ball through snow and fog, and maintain consistent judgment when human vision is compromised.

There’s also the question of unconscious bias. Recent studies have raised concerns about officiating consistency favoring certain teams. Visual AI offers the promise of more objective decision-making, not to replace referees, but to give them better tools to make the right call.

It’s important to emphasize that visual AI isn’t about replacing officials. It’s about augmenting human judgment with better information. The human-in-the-loop approach keeps experienced referees in control while giving them access to insights that weren’t previously possible. Think of it as the difference between a doctor making a diagnosis based solely on physical examination versus having access to MRI scans and blood work. The doctor still makes the call, but they’re making it with better information.

Why Visual AI Will Be a Game Changer

Today’s NFL games deploy vastly more cameras than just a decade ago. Beyond the increase in quantity, we now have specialized cameras: super slow-motion rigs capturing thousands of frames per second, robotic systems that can reposition instantly, handheld cameras for intimate player perspectives, and alternate angle views that capture every inch of the field. Visual AI can integrate and synthesize all these feeds simultaneously.

Consider replay reviews, which currently can take several minutes as officials review footage from multiple angles sequentially. Visual AI can synchronize multi-angle video, automate ball positioning with precision down to the inch, and deliver conclusions in seconds rather than minutes, even if that conclusion is simply finding the best clip and angle for the officiating crew to study. This speeds up the game while improving accuracy.

One particularly promising application involves tracking when the ball crosses the sideline during kickoffs and punts. Currently, officials make subjective estimations about ball placement. By synchronizing and instantly analyzing multi-camera perspectives, visual AI can dramatically improve both the speed and reliability of these spot placements.

For fans at home, the possibilities are equally exciting. Imagine player names dynamically superimposed on your screen, updating as the camera follows the action. Picture real-time views of what individual players see from their perspective on the field or probability overlays showing the likelihood of different play outcomes based on formation, down, distance, and historical data.

Perhaps most intriguingly, visual AI can show viewers real-time analytics on close calls and questionable rulings as they happen. Rather than waiting for the halftime show to debate a controversial call, fans could see the AI’s analysis immediately, complete with confidence levels and supporting evidence from multiple angles.

Visual AI represents more than just better instant replay or fancier graphics. It’s about fundamentally enhancing the game for everyone involved. Players benefit from more accurate officiating. Coaches gain better tools for reviewing and contesting calls. Referees receive support that helps them make correct decisions under immense pressure. Fans enjoy a richer, more engaging viewing experience with deeper insights into the game they love. 

The Super Bowl has always been a showcase, not just of football, but of what is possible when sports, culture, and technology collide. As visual AI continues to mature, it has the potential to set a new standard for how games are played, officiated, and experienced. In a sport defined by inches, seeing more clearly can make all the difference.

The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.

This story was originally featured on Fortune.com