Matthew Gombolay sees a future for human-scale robots in sports and athletic training.

He imagines robots that can test the skills of professional athletes as well as novices looking to learn a new sport. His latest invention may serve as the touchstone to getting there.

Gombolay grew up playing a variety of sports, but none appealed to him more than tennis. He said he had been playing with the idea of constructing a tennis robot that could go beyond training against a stationary ball feeder to help a player improve his skills.

[VIDEO: See Georgia Tech's Tennis Robot In Action]

What if he could have a tennis partner that could play with or against him any time he wanted and could help him improve the weakest areas of his game or complement him in a game of doubles?

“Right now, you use ball machines to hit balls of various spin speeds and locations that can emulate what a match might look like,” Gombolay said. “But that’s very different than accounting for an opponent moving across the courts who is going to be hitting from different positions and with different capabilities in shot selection.”

But what would be the best way to create a robot that could simulate a human opponent? He found the answer in one of the sport’s offshoots — wheelchair tennis.

A rapidly growing sport, wheelchair tennis now has its own professional league and is played at all four Grand Slam tournaments as well as the Summer Paralympics.

“We bought a wheelchair that’s designed for wheelchair tennis because I thought bi-pedal locomotion was a little beyond us at this point,” Gombolay said. “If we had an Atlas robot stomping around on the court, we would damage the court.”

An associate professor of robotics in the School of Interactive Computing, Gombolay has spent the past two years building his passion project. It’s called ESTHER — a wheelchair tennis robot that has a tennis racket connected to a single arm. It can cover both sides of the court and could potentially change how robotics can enhance athletic training and performance.

“What really excites me is that it could be a partner for me one day,” Gombolay said. “It can also be my opponent. It can help me train. I could have it pretend to be the one guy I always lose to because he can exploit this weakness in my game.

“Training against an opponent is psychologically more stressful. Getting closer to simulating real match conditions can help you improve performance.”

ESTHER stands for Experimental Sport Tennis Wheelchair Robot, and its name is a homage to renowned wheelchair tennis player Esther Vergeer. Vergeer held the world No. 1 ranking in women’s wheelchair tennis from 1999 to when she retired in 2013, winning 48 major titles and seven Paralympic gold medals. She was 695-25 in singles matches over her career.

ESTHER is nowhere close to the skill level of the player its namesake, but building a human-scale robot that can hit a return is a novel achievement.

Working with more than 20 students, Gombolay authored a paper on building ESTHER, which was accepted to be published in the Institute of Electrical and Electronics Engineers Robotics & Automation Society’s Automation Letters (IEEE RA-L). The team reached a breakthrough late last year when they successfully and consistently programmed ESTHER to locate the tennis ball coming toward it, and to hit a return.

“It took us about two years to get to that point because nobody's done this before,” Gombolay said. “We built this up from the ground up. Developing that capability was truly exciting.”

ESTHER is powered by two DC motors connected to a gearbox, giving it the quick burst it needs to roll from one side of the court to the other.

“Getting to the ball is not the problem,” Gombolay said. “The problem is knowing where to go. You have to know so far in advance to figure out where the robot should go. There are some problems for us in pathfinding — both deciding where to intercept the ball and what path to get there and being able to follow that trajectory with the robot.”

Gombolay and his team arranged a network of high-resolution cameras around a tennis court and used computer vision algorithms to help ESTHER recognize an incoming tennis ball. The team uses an orange tennis ball because a yellow or green ball can easily get lost among the colors of the court and other surrounding objects.

“When we use cameras from different angles, we can triangulate where the ball is in space,” said Nathaniel Belles, a master’s student in robotics who works on Gombolay’s team. “And once we have enough samples, over time, we can use that position of the ball and see what the trajectory of the ball is, where the arc is going, and where it’s going to end up.”

While Gombolay and his team have a long-term vision for ESTHER, it’s a project full of baby steps. The team is now working toward making ESTHER capable of hitting a back-and-forth rally.

Once it can do that, the next phase would be teaching it how to strategize its shot selection.

“We’re working on important modifications like control methods to control where a robot will go,” Gombolay said. “We’re using reinforcement learning methods so that the robot can learn by itself to get better at where it should go and how it should hit the ball. It should start getting more aggressive and thinking about how it wants to hit the ball and where it wants to hit the ball to win the game.”

If it can reach that point, the goal of revolutionizing the way athletes prepare and train could be within reach.

“From a training perspective, a system like this could one day help you to play or practice against your opponent, the robot playing as your opponent, adopting the same hitting strategies or tactics without you ever having to actually go and play that opponent,” Gombolay said. “After you’ve honed your game against your opponent, then you can go play and hopefully win.”

Zulfiqar Zaidi, one of the lead students on the project, said the benefits of ESTHER’s technology could expand beyond tennis.

“While tennis is a great starting point, a system that can play tennis well can have applications in other fields that similarly require fast dynamic movements, accurate perception, and the ability to safely move around humans,” Zaidi said. “This technology could be useful in manufacturing, construction, or any other field that requires a robot to interact with humans while performing fast and precise movements.

“The ability to plan and strategize well ahead of time is critical for successful performance in both tennis and other fields. A system that can do this effectively for tennis can also be applied to other scenarios where it is necessary to consider the effect of the current action on the future state of the system.”