Every sports fan can recall a moment that had them screaming from their seat or their couch as their favorite team landed on the wrong end of an official’s crucial call. Now, artificial intelligence could offer assistance.

Ayush Pai, an avid NBA viewer, set out to create a solution as he watched fouls go uncalled and outcomes be decided by officials rather than the players.

With a self-taught knowledge of computer vision AI, he created the initial model for the official­­ –– dubbed version 1.0 –– in the summer of 2022 after graduating high school. With soccer's video assistant referee (VAR) system and the Hawk-Eye review system in tennis as a guide, Pai felt that basketball was missing an AI component to supplement referees on the court.

"The officials are running up and down the court constantly. Basketball in general, especially the NBA, is one of the most fast-paced sports, and the entire court is difficult for officials to see at all times. Why not implement a system that can enhance their ability to see the floor using multiple cameras with real-time processing and detection of the basketball and the players?" he said.

Version 1.0 of Pai's system was trained to detect the color of the basketball and used a pedometer to track steps, but lighting issues and the use of a cumbersome step counter proved limiting. In June, the Denver Nuggets capped a 4-1 series victory over the Miami Heat to earn their first NBA championship, but officiating mistakes in the Heat's lone win propelled Pai to get back to work on his concept. By the end of the month, version 2.0 of the AI official was complete.

To improve accuracy, he used over 3,000 photos of basketballs to better train the model to track the ball's movement, and he also used pose estimation for tracking the position of a player's ankles and wrists to detect double dribbling and traveling violations. He is now working on version 3.0, which will include the capacity for multiple players, cameras, and the ability to detect shooting fouls and reach-ins. Pai is also developing a mobile app that he hopes will bring accurate officiating to youth leagues and pickup games worldwide.

The AI craze shows no signs of slowing down, and Pai sees his creation as a way to introduce the concept to new audiences. "Everyone should see the potential of AI, and it's important that everyone has a basic understanding of its implications and how it can be used in different fields. You can use AI for ChatGPT, in Tesla's autonomous systems, and even in sports," he said.

An ongoing debate amid the rise of AI is its potential to displace people. Pai sees the benefit of his system as one of coexistence rather than a replacement for the human element of officiating.

"You don't want to make AI feel like a threat to the people in any industry. It's a much better approach to consider this a partnership between AI and officials. In basketball, there are so many physical aspects of the job that AI can't do," he said.

The Florida native will trade his University of Florida Gator blue and orange for Yellow Jacket white and gold this fall as a transfer student, and the incoming second-year student is eager to get started.  

"One of the main reasons I transferred is because of the opportunities the Institute offers within AI and computer science,” he said. “The chance to participate in the College of Computing's Threads program will help me with my projects, and I’m looking forward to exploring the entrepreneurial side of Tech with great programs that foster innovation." 

Pai’s interest in the capabilities of AI isn’t limited to basketball. He has also created systems that attempt to discourage texting and driving and keep students on task while studying.

Georgia Tech continues to cement itself as a leader in the emerging space, highlighted by the recent launch of the AI Hub on campus, which will drive education and research while developing real-world, responsible applications for AI.