Indoor farms, also known as vertical farms, are popular among agricultural researchers and are expanding across the agricultural industry. Some benefits they have over outdoor farms include:
Additionally, some studies indicate that indoor farms produce more nutritious food for urban communities.
However, these farms are often inaccessible to birds, bees, and other natural pollinators, leaving the pollination process to humans. The tedious process must be completed by hand for each flower to ensure the indoor crop flourishes.
Ai-Ping Hu, a principal research engineer at the Georgia Tech Research Institute (GTRI), has spent years exploring methods to efficiently pollinate flowering plants and food crops in indoor farms to find a way to efficiently pollinate flower plants and food crops in indoor farms.
Hu, Assistant Professor Shreyas Kousik of the George W. Woodruff School of Mechanical Engineering, and a rotating group of student interns have developed a robot prototype that may be up to the task.
The robot can efficiently pollinate plants that have both male and female reproductive parts. These plants only require pollen to be transferred from one part to the other rather than externally from another flower.
Natural pollinators perform this task outdoors, but Hu said indoor farmers often use a paintbrush or electric tootbrush to ensure these flowers are pollinated.
An early challenge the research team addressed was teaching the robot to identify the “pose” of each flower. Pose refers to a flower’s orientation, shape, and symmetry. Knowing these details ensures precise delivery of the pollen to maximize reproductive success.
“It’s crucial to know exactly which way the flowers are facing,” Hu said.
“You want to approach the flower from the front because that’s where all the biological structures are. Knowing the pose tells you where the stem is. Our device grasps the stem and shakes it to dislodge the pollen.
“Every flower is going to have its own pose, and you need to know what that is within at least 10 degrees.”
Harsh Muriki is a robotics master’s student at Georgia Tech’s School of Interactive Computing, who used computer vision to solve the pose problem while interning for Hu and GTRI.
Muriki attached a camera to a FarmBot to capture images of strawberry plants from dozens of angles in a small garden in front of Georgia Tech’s Food Processing Technology Building. The FarmBot is an XYZ-axis robot that waters and sprays pesticides on outdoor gardens, though it is not capable of pollination.
“We reconstruct the images of the flower into a 3D model and use a technique that converts the 3D model into multiple 2D images with depth information,” Muriki said. “This enables us to send them to object detectors.”
Muriki said he used a real-time object detection system called YOLO (You Only Look Once) to classify objects. YOLO is known for identifying and classifying objects in a single pass.
Ved Sengupta, a computer engineering major who interned with Muriki, fine-tuned the algorithms that converted 3D images into 2D.
“This was a crucial part of making robot pollination possible,” Sengupta said. “There is a big gap between 3D and 2D image processing.
“There’s not a lot of data on the internet for 3D object detection, but there’s a ton for 2D. We were able to get great results from the converted images, and I think any sector of technology can take advantage of that.”
Sengupta, Muriki, and Hu co-authored a paper about their work that was accepted to the 2025 International Conference on Robotics and Automation (ICRA) in Atlanta.
The pollination robot, built in Kousik’s Safe Robotics Lab, is now in the prototype phase.
Hu said the robot can do more than pollinate. It can also analyze each flower to determine how well it was pollinated and whether the chances for reproduction are high.
“It has an additional capability of microscopic inspection,” Hu said. “It’s the first device we know of that provides visual feedback on how well a flower was pollinated.”
For more information about the robot, visit the Safe Robotics Lab project page.
]]>A Georgia Tech research led by Ai-Ping Hu and Shreyas Kousik team is working to solve that. A robot they've developed can efficiently pollinate plants that have both male and female reproductive parts. These plants only require pollen to be transferred from one part to the other rather than externally from another flower.
]]>In these contests, a handler directs a trained dog with whistle signals to guide a small group of sheep across a field and sometimes split the flock cleanly into two groups. But sheep do not always cooperate.
Researchers at the Georgia Institute of Technology studied how handler–dog teams manage these unpredictable flocks in sheepdog trials and found principles that extend beyond livestock herding.
In a study published in Science Advances as the cover feature, the researchers applied those insights to computer simulations showing how similar strategies could improve the control of robot swarms, autonomous vehicles, AI agents, and other networked systems where many machines must coordinate their actions despite uncertain conditions.
Group Movement Dynamics
“Birds, bugs, fish, sheep, and many other organisms move in groups because it benefits individuals, including protection from predators,” said Saad Bhamla, an associate professor in Georgia Tech’s School of Chemical and Biomolecular Engineering. “The puzzle is that the ‘group’ is not a single organism. It is built from many individuals, each making local, imperfect decisions.”
When a predator threatens a herd of sheep, individuals near the edge often move toward the center to reduce their own risk, Bhamla explained. “This is ‘selfish herd’ behavior,” he said. “Shepherds exploit that instinct using trained dogs.”
From examining hours of contest footage, the researchers found that controlling small groups of sheep can be harder than managing large ones. A larger group, with more sheep protected in the center, may behave more coherently than a small group as the animals constantly shift between two instincts: “follow the group” and “flee the dog.”
“That switching behavior makes the group unpredictable,” said Tuhin Chakrabortty, a former postdoctoral researcher in the Bhamla Lab who co-led the study.
Looking closely at how dogs and their handlers guide small groups, the researchers found that unpredictability in the flock’s behavior does not always make control harder. “Under the right conditions, that ‘noisy’ behavior might actually be a benefit,” Bhamla said.
Successful Sheep Herding
Sheepdog handlers categorize sheep by how strongly they respond to a dog’s threatening pressure. Some very responsive sheep might panic under too much pressure, while others might ignore mild pressure and require stronger positioning by the dog.
The researchers observed that successful control often followed a two-step pattern. First, the dog subtly influenced the sheep’s orientation while the animals were mostly standing still. Once the flock was aligned in the desired direction, the dog increased pressure to trigger movement. The timing of those actions was critical, because alignment within a small group could disappear quickly as individuals switched between instincts.
“In our simulations, increasing pressure makes the flock reach the desired orientation faster, but how long the flock stays aligned is set mainly by noise,” Chakrabortty said. “In essence, dogs can steer the direction, but they can’t hold that decision indefinitely, so timing matters.”
Developing Computer Models
To understand the broader implications of that behavior, the team developed computer models that captured how sheep respond both to the dog and to one another. The models allowed the researchers to test different strategies for guiding groups whose members make independent decisions under uncertainty.
They then applied those ideas to simulations of robotic swarms. Engineers often design such systems so that each robot blends signals from all nearby robots before deciding how to move. While that approach works well when signals are clear, it can break down when information is noisy or conflicting, Bhamla explained.
To explain why that switching strategy can work under noisy conditions, the researchers used an analogy of a smoke-filled room where only one person can see the exit, and no one knows who that person is. If everyone polls everyone else and averages the guesses, the one correct signal can get diluted by many noisy ones.
“That’s the counterintuitive part. When only one person has the right information, averaging can wash out the signal. But if you follow one person at a time, and keep switching who that is, the right information can spread through the crowd,” Bhamla said.
Building on that idea, the researchers tested a strategy inspired by the switching behavior they observed in sheep. In the simulations, each robot paid attention to just one source at a time (either a guiding signal or a neighboring robot) and switched that source from one step to the next.
Under noisy conditions, this switching strategy required less effort to keep the group moving along a desired path than either averaging-based strategies or fixed leader-follower strategies.
The researchers call their approach the Indecisive Swarm Algorithm. The name reflects a counterintuitive insight: allowing influence to shift among individuals over time can make groups easier to guide when conditions are uncertain.
“Our findings suggest that the same dynamics that make small animal groups unpredictable may also offer new ways to control complex engineered systems,” Bhamla said.
CITATION: Tuhin Chakrabortty and Saad Bhamla, “Controlling noisy herds: Temporal network restructuring improves control of indecisive collectives,” Science Advances, 2026
This research was funded in part by Schmidt Sciences as part of a Schmidt Polymath grant to Saad Bhamla.
Even when engineers have a working device and brain, called a controller, changes and improvements to the exoskeleton system typically mean data collection and training start all over again. The process is expensive and makes bringing fully functional exoskeletons or robotic limbs into the real world largely impractical.
Not anymore, thanks to Georgia Tech engineers and computer scientists.
They’ve created an artificial intelligence tool that can turn huge amounts of existing data on how people move into functional exoskeleton controllers. No data collection, retraining, and hours upon hours of additional lab time required for each specific device.
Their approach has produced an exoskeleton brain capable of offering meaningful assistance across a huge range of hip and knee movements that works as well as the best controllers currently available. Their worked was published Nov. 19 in Science Robotics.
Full details on the College of Engineering website.
]]>World foundation models (WFMs) enable physical AI systems to learn and operate within synthetic worlds created by generative artificial intelligence (genAI). For example, these models use predictive capabilities to generate up to 30 seconds of video that accurately reflects the real world.
The new framework, developed by a Georgia Tech researcher, enhances the processing speed of the neural networks that simulate these real-world environments from text, images, or video inputs.
The neural networks that make up the architectures of large language models like ChatGPT and visual models like Sora process contextual information using the “attention mechanism.”
Attention refers to a model’s ability to focus on the most relevant parts of input.
The Neighborhood Attention Extension (NATTEN) allows models that require GPUs or high-performance computing systems to process information and generate outputs more efficiently.
Processing speeds can increase by up to 2.6 times, said Ali Hassani, a Ph.D. student in the School of Interactive Computing and the creator of NATTEN. Hassani is advised by Associate Professor Humphrey Shi.
Hassani is also a research scientist at Nvidia, where he introduced NATTEN to Cosmos — a family of WFMs the company uses to train robots, autonomous vehicles, and other physical AI applications.
“You can map just about anything from a prompt or an image or any combination of frames from an existing video to predict future videos,” Hassani said. “Instead of generating words with an LLM, you’re generating a world.
“Unlike LLMs that generate a single token at a time, these models are compute-heavy. They generate many images — often hundreds of frames at a time — so the models put a lot of work on the GPU. NATTEN lets us decrease some of that work and proportionately accelerate the model.”
]]>A new study explains how tiny water bugs use fan-like propellers to zip across streams at speeds up to 120 body lengths per second. The researchers then created a similar fan structure and used it to propel and maneuver an insect-sized robot.
The discovery offers new possibilities for designing small machines that could operate during floods or other challenging situations.
Instead of relying on their muscles, the insects about the size of a grain of rice use the water’s surface tension and elastic forces to morph the ribbon-shaped fans on the end of their legs to slice the water surface and change directions.
Once they understood the mechanism, the team built a self-deployable, one-milligram fan and installed it into an insect-sized robot capable of accelerating, braking, and maneuvering right and left.
The study is featured on the cover of the journal Science.
Read the entire story and see the robot in action on the College of Engineering website.
The discovery offers new possibilities for designing small machines that could operate during floods or other challenging situations.
]]>He also has cerebral palsy — and for the past four years, he has played a key role in improving one of the most exciting medical devices at Georgia Tech.
“My role here is as a participant in exoskeleton research studies,” Case explained. “When I come in, researchers hook me up to sensors that monitor my gait when I’m walking in the device, and then they get a whole lot of data based off that.”
]]>A team of Georgia Tech researchers wanted to ease that struggle, and robotic exoskeletons offered a promising path. Their findings point to a simple but powerful shift: exoskeletons that adapt to people, rather than forcing people to adapt to the machine. Using artificial intelligence (AI) to learn the rhythm of patients’ strides in real time, the team showed how these devices can reduce strain and increase efficiency. They also demonstrated how the technology can help restore confidence for stroke survivors.
The Robot Finds the Rhythm
A robotic exoskeleton is a wearable device that helps people move with mechanical support. Traditional exoskeletons require endless manual adjustments — turning knobs, calibrating settings, and tweaking controls.
“It can be frustrating, even nearly impossible, to get it right for each person,” said Aaron Young, associate professor in the George W. Woodruff School of Mechanical Engineering. “With AI, the exoskeleton figures out the mapping itself. It learns the timing of someone’s gait through a neural network, without an engineer needing to hand-tune everything.”
The software monitors each step, instantly updates, and fine-tunes the support it provides. Over time, the exoskeleton aligns its movements with the unique gait of the person wearing it. In this study, the research team used a hip exoskeleton, which provides torque at the hip joint — in other words, adding power to help stroke survivors walk or move their legs more easily.
Taking Smarter Steps
Walking after a stroke can be tough and unpredictable. A patient’s stride can change from one day to the next, and even from one step to the next. Most exoskeletons aren’t built for that kind of variation. They are designed around the steady, even gait of healthy young adults, which can leave stroke survivors feeling more unsteady than supported.
Young’s breakthrough, detailed in IEEE Transactions on Robotics, is a neural network — a type of AI that learns patterns much like the human brain does. Sensors at the hip pick up how someone is moving, and the network translates those signals into just the right boost of power to support each step. It quickly figures out a person’s unique walking pattern. But lead clinician Kinsey Herrin said the AI’s learning doesn’t stop there. It keeps adjusting as the patient walks, so the exoskeleton can stay in sync even during stride shifts.
“The speed really surprised us,” Young said. “In just one to two minutes of walking, the system had already learned a person’s gait pattern with high accuracy. That’s a big deal, to adapt that quickly and then keep adapting as they move.”
Tests showed the system was far more accurate than the standard exoskeleton. It reduced errors in tracking stroke patients’ walking patterns by 70%.
Young emphasized that this research is about more than metrics. “When you see someone able to walk farther without becoming exhausted, that’s when you realize this isn’t just about robotics — it’s about giving people back a measure of independence,” he said.
Adapting Anywhere
Every exoskeleton comes with its own set of sensors, so the data they collect can look completely different from one device to the next. A neural network trained on one machine often stumbles when it’s moved to another. To get around that, Young’s team designed software that works like a universal adapter plug — no matter what device it’s connected to, it converts the signals into a form the AI can use. After just 10 strides of calibration, the system cut error rates by more than 75%.
“The goal is that someone could strap on a device, and, within a minute, it feels like it was built just for them,” Young said.
A Step Toward the Future
While the study centered on stroke survivors, the implications are far broader. The same adaptive approach could support older adults coping with age-related muscle weakness, people with conditions like Parkinson’s or osteoarthritis, or even children with neurological disabilities.
Young and his team are now running clinical trials to measure how well the AI-powered exoskeleton supports people in a wide range of everyday activities.
“There’s no such thing as an ‘average’ user,” Young said. “The real challenge is designing technology that can adapt to the full spectrum of human mobility.”
If Georgia Tech’s exoskeleton can rise to that challenge, the promise goes well beyond the lab. It could mean a world where technology doesn’t just help people walk — it learns to walk with them.
Inseung Kang, who holds a B.S., M.S., and Ph.D. from Georgia Tech, is the paper’s lead author and now an assistant professor of mechanical engineering at Carnegie Mellon University. He explained that the real promise is in what comes next.
“We’ve developed a system that can adjust to a person’s walking style in just minutes. But the potential is even greater. Imagine an exoskeleton that keeps learning with you over your lifetime, adjusting as your body and mobility change. Think of it as a robot companion that understands how you walk and gives you the right assistance every step of the way.”
Aaron Young is affiliated with Georgia Tech’s Institute for Robotics and Intelligent Machines.
This research was primarily funded by a grant (DP2HD111709-01) from the National Institutes of Health New Innovator Award Program. Georgia Tech researchers have created the first lung-on-a-chip with a functioning immune system, allowing it to respond to infections much like a real human lung. The breakthrough, published in Nature Biomedical Engineering, provides a more accurate way to study diseases, test therapies, and reduce reliance on animal models. With potential applications in conditions from influenza to cancer, the technology opens the door to personalized medicine that predicts how individual patients will respond to treatment.
]]>Danfei Xu, an assistant professor in Georgia Tech’s School of Interactive Computing, is introducing new models that provide robots with “on-the-job” training.
The National Science Foundation (NSF) awarded Xu its CAREER award given to early career faculty. The award will enable Xu to expand his research and refine his models, which could accelerate the process of robot deployment and alleviate manufacturers from the burden of achieving perfection.
“The main problem we’re trying to tackle is how to allow robots to learn on the job,” Xu said. “How should it self-improve based on the performance or the new requirements or new user preferences in each home or working environment? You cannot expect a robot manufacturer to program all of that.
“The challenging thing about robotics is that the robot must get feedback from the physical environment. It must try to solve a problem to understand the limits of its abilities so it can decide how to improve its own performance.”
As with humans, Xu views practice as the most effective way for a robot to improve a skill. His models train the robot to identify the point at which it failed in its task performance.
“It identifies that skill and sets up an environment where it can practice,” he said. “If it needs to improve opening a drawer, it will navigate itself to the drawer and practice opening it.”
The models allow the robot to split tasks into smaller parts and evaluate its own skill level using reward functions. Cooking dinner, for example, can be divided into steps like turning on the stove and opening the fridge, which are necessary to achieve the overall goal.
“Planning is a complex problem because you must predict what’s going to happen in the physical world,” Xu said. “We use machine learning techniques that our group has developed over the past two years, using generated models to generate positive futures. They’re very good at modeling long-horizon phenomena.
“The robot knows when it’s failed because there’s a value that tells it how well it performed the task and whether it received its reward. While we don’t know how to tell the robot why it failed, we have ways for it to improve its skills based on that measurement.”
One of the biggest barriers that keeps many robots from being made available for public use is the pressure on manufacturers to make the robot as close to perfect as possible at deployment. Xu said it’s more practical to accept that robots will have learning gaps that need to be filled and to implement more efficient real-world learning models.
“We work under the pressure of getting everything correct before deployment,” he said. “We need to meet the basic safety requirements, but in terms of competence, it is difficult to get that perfect at deployment. This takes some of the pressure off because it will be able to self-adapt.”
Yalong Yang, another assistant professor in the School of IC, also received the NSF CAREER Award for a research proposal that will design augmented and virtual reality (AR/VR) workspaces for data workers.
“In 10 years, I envision everyone will use AR/VR in their office, and it will replace their laptop or their monitor,” Yang said.
Yang said he is also working with Google on the project and using Google Gemini to bring conventional applications to immersive space, with data tools being the most complicated systems to re-design for immersive environments.
The immersive workspace and interface will also enable teams of data workers to collaborate and share their data in real-time.
“I want to support the end-to-end process,” Yang said. “We have visualization tools for data, but it’s not enough. Data science is a pipeline — from collecting data to processing, visualizing, modeling and then communicating. If you only support one, people will need to switch to other platforms for the other steps.”
Yang also noted that prior research has shown that VR can enhance cognitive abilities, such as memory and attention and support multitasking. The results of his project could lead to maximizing worker efficiency without them feeling strained.
“We all have a cognitive limit in our working memory. Using AR/VR can increase those limits and process more information. We can expand people’s spatial ability to help them build a better mental model of the data presented to them.”
Yang was also recently named a 2025 Google Research Scholar as he seeks to build a new artificial intelligence (AI) tool that converts mobile apps into 3D immersive environments.
]]>Before launching its prototype, a research team within Georgia Tech’s School of Interactive Computing, led by Professor Bruce Walker and Assistant Professor Sehoon Ha, is working to improve its methods and designs based on research within blind and visually impaired (BVI) communities.
“There’s been research on the technical aspects and functionality of robotic guide dogs, but not a lot of emphasis on the aesthetics or form factors,” said Avery Gong, a recent master’s graduate who worked in Walker’s lab. “We wanted to fill this gap.”
Training a guide dog can cost up to $50,000, and while there are nonprofit organizations that can cover these costs for potential owners, there is still a gap between the amount of available guide dogs and BVI individuals who need them. Not all BVI individuals are able to care for a dog and feed it. The dog also has fewer than 10 working years before it needs replacement.
Gong co-authored a paper on the design implications of the robotic guide dog that was presented at the 2025 International Conference on Robotics and Automation (ICRA) in Atlanta in May.
The consensus among the study’s participants indicates they prefer a robotic guide dog that:
“A lot of people said they didn’t want the dog to look too cute or appealing because it would draw too much attention,” said Aviv Cohav, another lead author of the paper and recent master’s graduate.
“Many people have issues with taking their guide dog to places, whether it’s little kids wanting to play with the dog or people not liking dogs or people being scared of them, and that reflects on the owners themselves. We wanted to look at what would be a good balance between having a functional robot that wouldn’t scare people away or be a distraction.”
The researchers also had to consider the perspectives of sighted individuals and how society at large might view a robotic guide dog.
An example of this is the amount of noise the dog makes while walking. The owner needs to hear the dog is active, but the clanky sound many off-the-shelf robots make could create disturbances in indoor spaces that amplify sounds. To offset the noise, the team developed algorithms that allow the robot to move more quietly.
Walker and his lab have examined similar scenarios that must take public perception into account.
“We like to think of Georgia Tech as going the extra mile,” Walker said. “Let’s not just make a robot, but a robot that’s going to fit into society.
“To have impact, the technologies we produce must be produced with society in mind. This is a holistic design that considers the users and all the people with whom the users interact.”
Taery Kim, a computer science Ph.D. student, began working on the concept of a robotic guide dog when she came to Georgia Tech in 2022. She and Ha, her advisor, have authored papers on building the robot’s navigation and safety components.
“When I started, I thought it would be as simple as giving the guide dog a command to take me to Starbucks or the grocery store, and it would just take me,” Kim said. “But the user must give waypoint directions — ‘go left here,’ ‘turn right,’ ‘go forward,’ ‘stop.’ Detailed commands must be delivered to the dog.”
While a real dog has naturally enhanced senses of hearing and smell that can’t be replicated, technology can provide interconnected safety features during an emergency. The researchers envision a camera system equipped with a 360-degree field of view, computer vision algorithms that detect obstacles or hazards, and voice recognition that recognizes calls for help. An SOS function could automatically call 911 at the owner’s request or if the owner is unresponsive.
Kim said the robot should also have explainability features to enhance communication with the owner. For example, if the robot suddenly stops or ignores an owner’s commands, it should tell the owner that it’s detecting a hazard in their path.
Manufacturing a robot at scale would initially be expensive, but the researchers believe the cost would eventually be offset because of its longevity. BVI individuals may only need to purchase one during their lifetime.
To introduce a prototype, the multidisciplinary research team recognizes that it needs to enlist experts from other fields to adequately address the various implications and research gaps inherent in the project.
Walker said the teams welcome additional partners who are keen to tackle challenges ranging from design and engineering to battery life to human-robot interaction.
Team member J. Taery Kim was supported by the National Science Foundation's Graduate Research Fellowship Program (NSF GRFP) under Grant No. DGE-2039655.
]]>nathan.deen@cc.gatech.edu
]]>Led by PI: Taylor Higgins, Assistant Professor, FAMU-FSU Department of Mechanical Engineering, partnering with Co-PIs Shreyas Kousik, Assistant Professor, Georgia Tech, George W. Woodruff School of Mechanical Engineering, and Brady DeCouto, Assistant Professor, FSU Anne Spencer Daves College of Education, Health, and Human Sciences, the research will use the acquisition of unicycle riding skill by participants to gain a better grasp on human motor learning in tasks requiring balance and complex movement in space. Although it might sound a bit odd, the fact that most people don’t know how to ride a unicycle, and the fact that it requires balance, mean that the data will cover the learning process from novice to skilled across the participant pool.
Using data acquired from human participants, the team will develop a “robotics assistive unicycle” that will be used in the training of the next pool of novice unicycle riders. This is to gauge if, and how rapidly, human motor learning outcomes improve with the assistive unicycle. The participants that engage with the robotic unicycle will also give valuable insight into developing effective human-robot collaboration strategies.
The fact that deciding to get on a unicycle requires a bit of bravery might not be great for the participants, but it’s great for the research team. The project will also allow exploration into the interconnection between anxiety and human motor learning to discover possible alleviation strategies, thus increasing the likelihood of positive outcomes for future patients and consumers of these devices.
Author
-Christa M. Ernst
This Article Refers to NSF Award # 2449160
]]>Georgia Tech roboticists are attempting to identify the most stressful periods that human teleoperators experience while performing tasks remotely. A novel study provides new insights into determining when a teleoperator needs to operate at a high level of focus and which parts of the task can be delegated to robot automation.
School of Interactive Computing Associate Professor Matthew Gombolay calls it the “sweet spot” of human ingenuity and robotic precision. Gombolay and students from his CORE Robotics Labconducted a novel study that measures stress and workload on human teleoperators.
Gombolay said it can inform military officials on how to strategically implement task automation and maximize human teleoperator performance.
Humans continue to hand over more tasks to robots to perform, but Gombolay said that some functions will still require human input and oversight for the foreseeable future.
Specific applications, such as space exploration, commercial and military aviation, disaster relief, and search and rescue, pose substantial safety concerns. Astronauts stationed on the International Space Station, for example, manually control robots that bring in supplies, move cargo, and make structural repairs.
“It’s brutal from a psychological perspective,” Gombolay said.
The question often asked about automating a task in these fields is, at what point can a robot be trusted more than a human?
A recent paper by Gombolay and his current and former students — Sam Yi Ting, Erin Hedlund-Botti, and Manisha Natarajan — sheds new light on the debate. The paper was published in the IEEE Robotics and Automation Letters and will be presented at the International Conference on Robotics and Automation in Atlanta.
The NASA-funded study can identify which aspects of tedious, time-consuming tasks can be automated and which require human supervision. If roboticists can pinpoint the elements of a task that cause the least stress, they can automate these components and enable humans to oversee the more challenging aspects.
“If we’re talking about repetitive tasks, robots do better with that, so if you can automate it, you should,” said Ting, a former grad student and lead author of the paper. “I don’t think humans enjoy doing repetitive tasks. We can move toward a better future with automation.”
Military officials, for example, could measure the stress of remote drone pilots and know which times during a pilot’s shift require the highest level of attention.
“We can get a sense of how stressed you are and create models of how divided your attention is and the performance rate of the tasks you’re doing,” Gombolay said.
“It can be a low-stress or high-stress situation depending on the stakes and what’s going on with you personally. Are you well-caffeinated? Well-rested? Is there stress from home you’re bringing with you to the workplace? The goal is to predict how good your task performance will be. If it indicates it might be poor, we may need to outsource work to other people or create a safe space for the operator to destress.”
For their study, the researchers cut a small river-shaped path into a medium-density fiberboard. The exercise required the 24 participants to use a remote robotic arm to navigate through the path from one end to the other without touching the edges.
The experiment grew more challenging as new stress conditions and workload requirements were introduced. The changing conditions required the test participants to multitask to complete the assignment.
Gombolay said the study supports the Yerkes-Dodson Law, which states that moderate levels of stress increase human performance.
The experiment showed that operators felt overwhelmed and performed poorly when multitasking was introduced. Too much stress led to poor performance, but a moderate amount of stress induced more engagement and enhanced teleoperator focus.
Ting said finding that ideal stress zone can lead to a higher performance rating.
“You would think the more stressed you are, the more your performance decreases,” Ting said. “Most people didn’t react that way. As stress increased, performance increased, but when you increased workload and gave them more to do, that’s when you started seeing deteriorating performance.”
Gombolay said no stress can be just as detrimental as too much stress. Performing a task without stress tends to cause teleoperators to become disinterested, especially if it is repetitive and time-consuming.
“No stress led to complacency,” Gombolay said. “They weren’t as engaged in completing the task.
“If your excitement is too low, you get so bored you can’t muster the cognitive energy to reason about robot operation problems.”
Roboticists have made significant leaps in recent years to remove teleoperators from the equation. Still, Gombolay said it’s too early to tell whether robots can be trusted with any task that a human can perform.
“We’re a long way from full autonomy,” he said. “There’s a lot that robots still can’t do without a human operator. Search and rescue operations, if a building collapses, we don’t have much training data for robots to go through rubble by themselves to rescue people. There are ethical needs for humans to be able to supervise or take direct control of robots.”
]]>Top South Korean research institutes have enlisted Georgia Tech researchers Sehoon Ha and Jennifer G. Kim to develop artificial intelligence (AI) to help the humanoid assistant navigate hospitals and interact with doctors, nurses, and patients.
Ha and Kim will partner with Neuromeka, a South Korean robotics company, on a five-year, 10 billion won (about $7.2 million US) grant from the South Korean government. Georgia Tech will receive about $1.8 million of the grant.
Ha and Kim, assistant professors in the School of Interactive Computing, will lead Tech’s efforts and also work with researchers from the Korea Advanced Institute of Science and Technology and the Electronics and Telecommunications Research Institute.
Neuromeka has built industrial robots since its founding in 2013 and recently decided to expand into humanoid service robots.
Lee, the group leader of the humanoid medical assistant project, said he fielded partnership requests from many academic researchers. Ha and Kim stood out as an ideal match because of their robotics, AI, and human-computer interaction expertise.
For Ha, the project is an opportunity to test navigation and control algorithms he’s developed through research that earned him the National Science Foundation CAREER Award. Ha combines computer simulation and real-world training data to make robots more deployable in high-stress, chaotic environments.
“Dr. Ha has everything we want to put into our system, including his navigation policies,” Lee said. “He works with robots and AI, and there weren’t many candidates in that space. We needed a collaborator who can create the software and has experience running it on robots.”
Ha said he is already considering how his algorithms could scale beyond hospitals and become a universal means of robot navigation in unstructured real-world environments.
“For now, we’re focusing on a customized navigation model for Korean environments, but there are ways to transfer the data set to different environments, such as the U.S. or European healthcare systems,” Ha said.
“The final product can be deployed to other systems and industries. It can help industrial workers at factories, retail stores, any place where workers can get overwhelmed by a high volume of tasks.”
Kim will focus on making the robot’s design and interaction features more human. She’ll develop a large-language model (LLM) AI system to communicate with patients, nurses, and doctors. She’ll also develop an app that will allow users to input their commands and queries.
“This project is not just about controlling robots, which is why Dr. Kim’s expertise in human-computer interaction design through natural language was essential.,” Lee said.
Kim is interviewing stakeholders from three South Korean hospitals to identify service and care pain points. The issues she’s identified so far relate to doctor-patient communication, a lack of emotional support for patients, and an excessive number of small tasks that consume nurses’ time.
“Our goal is to develop this robot in a very human-centered way,” she said. “One way is to give patients a way to communicate about the quality of their care and how the robot can support their emotional well-being.
“We found that patients often hesitate to ask busy nurses for small things like getting a cup of water. We believe this is an area a robot can support.”
The robot’s hardware will be built in Korea, while Ha and Kim will develop the software in the U.S.
Jong-hoon Park, CEO of Neuromeka, said in a press release the goal is to have a commercialized product as soon as possible.
“Through this project, we will solve problems that existing collaborative robots could not,” Park said. “We expect the medical AI humanoid robot technology being developed will contribute to reducing the daily work burden of medical and healthcare workers in the field.”
]]>Meta announced in February its partnership with the labs of professors Danfei Xu and Judy Hoffman on a novel computer vision-based algorithm called EgoMimic. It enables robots to learn new skills by imitating human tasks from first-person video footage captured by Meta’s Aria smart glasses.
Xu’s Robot Learning and Reasoning Lab (RL2) displayed EgoMimic in action at ICRA May 19-23 at the World Congress Center in Atlanta.
Lawrence Zhu, Pranav Kuppili, and Patcharapong “Elmo” Aphiwetsa — students from Xu’s lab — used Egomimic to compete in a robot teleoperation contest at ICRA. The team finished second in the event titled What Bimanual Teleoperation and Learning from Demonstration Can Do Today, earning a $10,000 cash prize.
Teams were challenged to perform tasks by remotely controlling a robot gripper. The robot had to fold a tablecloth, open a vacuum-sealed container, place an object into the container, and then reseal it in succession without any errors.
Teams completed the tasks as many times as possible in 30 minutes, earning points for each successful attempt.
The competition also offered different challenge levels that increased the points awarded. Teams could directly operate the robot with a full workstation view and receive one point for each task completion. Or, as the RL2 team chose, teams could opt for the second challenge level.
The second level required an operator to control the task with no view of the workstation except for what was provided to through a video feed. The RL2 team completed the task seven times and received double points for the challenge level.
The third challenge level required teams to operate remotely from another location. At this level, teams could earn four times the number of points for each successful task completed. The fourth level challenged teams to deploy an algorithm for task performance and awarded eight points for each completion.
Using two of Meta’s Quest wireless controllers, Zhu controlled the robot under the direction of Aphiwetsa, while Kuppili monitored the coding from his laptop.
“It’s physically difficult to teleoperate for half an hour,” Zhu said. “My hands were shaking from holding the controllers in the air for that long.”
Being in constant communication with Aphiwetsa helped him stay focused throughout the contest.
“I helped him strategize the teleoperation and noticed he could skip some of the steps in the folding,” Aphiwetsa said. “There were many ways to do it, so I just told him what he could fix and how to do it faster.”
Zhu said he and his team had intended to tackle the fourth challenge level with the EgoMimic algorithm. However, due to unexpected time constraints, they decided to switch to the second level the day before the competition due to unexpected time constraints.
“I think we realized the day before the competition training the robot on our model would take a huge amount of time,” Zhu said. “We decided to go for the teleoperation and started practicing.”
He said the team wants to tackle the highest challenge level and use a training model for next year’s ICRA competition in Vienna, Austria.
ICRA is the world’s largest robotics conference, and Atlanta hosted the event for the third time in its history, drawing a record-breaking attendance of over 7,000.
]]>“Popping the Manu” will make you a winner.
Georgia Tech researchers studied dives by the Māori, the indigenous people of New Zealand, who have made Manu jumping a cultural tradition. By hitting the water in a “V” shape, then quickly extending their bodies underwater, they’ve perfected the art of huge splashes.
See a video on how to make the splash and read the entire story on the College of Engineering homepage.
]]>Wang is a recipient of the 2025 Outstanding Dissertation Award from the Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI). The award recognizes Wang for his lifelong work on democratizing human-centered AI.
“Throughout my Ph.D. and industry internships, I observed a gap in existing research: there is a strong need for practical tools for applying human-centered approaches when designing AI systems,” said Wang, now a safety researcher at OpenAI.
“My work not only helps people understand AI and guide its behavior but also provides user-friendly tools that fit into existing workflows.”
[Related: Georgia Tech College of Computing Swarms to Yokohama, Japan, for CHI 2025]
Wang’s dissertation presented techniques in visual explanation and interactive guidance to align AI models with user knowledge and values. The work culminated from years of research, fellowship support, and internships.
Wang’s most influential projects formed the core of his dissertation. These included:
“I feel extremely honored and lucky to receive this award, and I am deeply grateful to many who have supported me along the way, including Polo, mentors, collaborators, and friends,” said Wang, who was advised by School of Computational Science and Engineering (CSE) Professor Polo Chau.
“This recognition also inspired me to continue striving to design and develop easy-to-use tools that help everyone to easily interact with AI systems.”
Like Wang, Chau advised Georgia Tech alumnus Fred Hohman (Ph.D. CSE 2020). Hohman won the ACM SIGCHI Outstanding Dissertation Award in 2022.
Chau’s group synthesizes machine learning (ML) and visualization techniques into scalable, interactive, and trustworthy tools. These tools increase understanding and interaction with large-scale data and ML models.
Chau is the associate director of corporate relations for the Machine Learning Center at Georgia Tech. Wang called the School of CSE his home unit while a student in the ML program under Chau.
Wang is one of five recipients of this year’s award to be presented at the 2025 Conference on Human Factors in Computing Systems (CHI 2025). The conference occurs April 25-May 1 in Yokohama, Japan.
SIGCHI is the world’s largest association of human-computer interaction professionals and practitioners. The group sponsors or co-sponsors 26 conferences, including CHI.
Wang’s outstanding dissertation award is the latest recognition of a career decorated with achievement.
Months after graduating from Georgia Tech, Forbes named Wang to its 30 Under 30 in Science for 2025 for his dissertation. Wang was one of 15 Yellow Jackets included in nine different 30 Under 30 lists and the only Georgia Tech-affiliated individual on the 30 Under 30 in Science list.
While a Georgia Tech student, Wang earned recognition from big names in business and technology. He received the Apple Scholars in AI/ML Ph.D. Fellowship in 2023 and was in the 2022 cohort of the J.P. Morgan AI Ph.D. Fellowships Program.
Along with the CHI award, Wang’s dissertation earned him awards this year at banquets across campus. The Georgia Tech chapter of Sigma Xi presented Wang with the Best Ph.D. Thesis Award. He also received the College of Computing’s Outstanding Dissertation Award.
“Georgia Tech attracts many great minds, and I’m glad that some, like Jay, chose to join our group,” Chau said. “It has been a joy to work alongside them and witness the many wonderful things they have accomplished, and with many more to come in their careers.”
]]>Wang is a recipient of the 2025 Outstanding Dissertation Award from the Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI). The award recognizes Wang for his lifelong work on democratizing human-centered AI.
]]>However, this new generation of self-thinking robots will need security protocols to ensure they aren’t susceptible to hackers. To integrate such robots into society, they must come with assurances that they will behave safely around humans.
It begs the question: can you guarantee the safety of something that doesn’t exist yet? It’s something Assistant Professor Glen Chou hopes to accomplish by developing algorithms that will enable autonomous systems to learn and adapt while acting with safety and security assurances.
He plans to launch research initiatives, in collaboration with the School of Cybersecurity and Privacy and the Daniel Guggenheim School of Aerospace Engineering, to secure this new technological frontier as it develops.
“To operate in uncertain real-world environments, robots and other autonomous systems need to leverage and adapt a complex network of perception and control algorithms to turn sensor data into actions,” he said. “To obtain realistic assurances, we must do a joint safety and security analysis on these sensors and algorithms simultaneously, rather than one at a time.”
This end-to-end method would proactively look for flaws in the robot’s systems rather than wait for them to be exploited. This would lead to intrinsically robust robotic systems that can recover from failures.
[RELATED: New Algorithm Teaches Robots Through Human Perspective]
Chou said this research will be helpful in other domains, including advanced space exploration. If a space rover is sent to one of Saturn’s moons, for example, it needs to be able to act and think independently of scientists on Earth.
Aside from fighting fires and exploring space, this technology could perform maintenance in nuclear reactors, automatically maintain the power grid, and make autonomous surgery safer. It could also bring assistive robots into the home, enabling higher standards of care.
This is a challenging domain where safety, security, and privacy concerns are paramount due to frequent, close contact with humans.
This will start in the newly established Trustworthy Robotics Lab at Georgia Tech, which Chou directs. He and his Ph.D. students will design principled algorithms that enable general-purpose robots and autonomous systems to operate capably, safely, and securely with humans while remaining resilient to real-world failures and uncertainty.
Chou earned dual bachelor’s degrees in electrical engineering and computer sciences as well as mechanical engineering from the University of California, Berkeley, in 2017, a master’s and Ph.D. in electrical and computer engineering from the University of Michigan in 2019 and 2022, respectively.
He was a postdoc at the Massachusetts Institute of Technology Computer Science & Artificial Intelligence Laboratory before joining Georgia Tech in November 2024. He received the National Defense Science and Engineering Graduate fellowship program, NSF Graduate Research fellowships, and was named a Robotics: Science and Systems Pioneer in 2022.
]]>Georgia Tech
School of Cybersecurity & Privacy
john.popham@cc.gatech.edu
]]>Georgia Tech’s Space Exploration and Analysis Laboratory (SEAL) has developed new algorithms that are headed to the Moon, as part of the Intuitive Machine’s IM-2 mission. The mission is sending a Nova-C class lunar lander named Athena to the Moon’s south pole region to test technologies and collect data that aim to enable future exploration. The mission is part of NASA’s Commercial Lunar Payload Services (CLPS) initiative.
SEAL, led by AE professor John Christian, collaborated with Intuitive Machines to develop algorithms to guide Athena to the Shackleton crater: a region known for its limited sunlight and cold temperatures. In coordination with SpaceX, launch of the company’s IM-2 mission is targeted for a multi-day launch window that opens no earlier than February 26 from Launch Complex 39A at NASA’s Kennedy Space Center in Florida.
Athena will transport NASA's PRIME-1 (Polar Resources Ice Mining Experiment-1) which includes two instruments: a drill and spectrometer. The Regolith and Ice Drill for Exploring New Terrain (TRIDENT) is designed to drill up to three feet of lunar surface to extract soil, while the mass spectrometer (MSOLO) will measure the amount of ice in the soil samples.
After launch, Athena will separate from the rocket and begin a roughly five-to-four-day cruise to the Moon’s orbit. The lander will orbit the Moon for approximately three to 1.5 days before its descent to the south pole.
In Fall 2022, Research Engineer Ava Thrasher (AE 2022, M.S. AE 2024) began working on IM-2, developing new algorithms to guide Athena to the Shackleton crater using optical terrain relative navigation (TRN). Her approach looked at developing a crater detection algorithm (CDA) using image processing techniques that capture crater center locations on the Moon which are then used to determine Athena's position estimations.
Then, she developed a crater identification algorithm (CIA) to match craters found in the image to a catalog of known lunar craters. By using CDA and CIA in tandem, Athena is able to estimate its location and orientation with a single photo, autonomously, and in real-time.
“We wanted to strike a balance between creating something that would be done quickly on board, but also something that was reliable,” she explained. “We ended up using simple crater geometry and knowledge of the sun angle to render what we expect a crater to look like in the image.”
The CDA finds craters by calculating a similarity score between the image and the rendered crater at each image pixel point. This process, also known as template matching, marks crater centers at points of very high similarity. CIA then uses these crater center locations to match them with known craters in a catalog. By matching pixel locations in an image to known three-dimensional positions on the Moon, the spacecraft is able to produce an estimation of its position.
After two years of research and testing, Thrasher, Christian, and the Intuitive Machines team successfully demonstrated the CDA and CIA on synthetic imagery and Thrasher handed off the algorithms to Intuitive Machines to convert them into flight software for Athena.
She first got involved with optical navigation (OPNAV) research after she took AE 4342: Senior Design with Prof. Christian as an undergraduate student. “I found optical navigation to be really interesting. I liked the idea of being able to figure out where you are and how you’re moving in real-time based on a picture,” she said. In Fall 2022, she started her first graduate semester at Tech and was a new member of SEAL, where she quickly began demonstrating the idea of detecting craters and prototyping the CDA and CIA programmed into Athena.
After she graduated with her master’s degree in aerospace engineering in May 2024, she loved what she did so much, that she decided to stay and work as a full-time research engineer in SEAL. Now, she’s gearing up to see her work make its way to the Moon.
“It's been really exciting and humbling to contribute to the massive task of putting a lander on the Moon. I never really appreciated the scale of work and collaboration needed to make it happen until I was lucky enough to be a part of it. I'll certainly be watching the launch and tracking the mission with great anticipation of both the engineering and scientific results,” said Thrasher.
As part of a multi-year collaboration, Christian helped develop a key navigation algorithm for Intuitive Machines’ first space mission (IM-1) which launched a Nova-C lunar lander named Odysseus to the Malapert A crater on the Moon’s south pole region; about 11 miles away from IM-2’s targeted Shackleton crater.
The IM-1 mission launched from Kennedy Space Center on February 15, 2024 and soft-landed on the Moon on February 22, 2024---making Odysseus the first U.S. lunar landing since the Apollo program and the first-ever successful commercial lunar landing. Odysseus had a rougher-than-expected soft landing due to an anomaly with the altimeter that was supposed to provide insight into the lander’s height above the lunar surface. In the absence of these altimeter measurements, Odysseus relied critically on the visual odometry technique that was jointly developed by Christian and Intuitive Machines.
Despite these challenges, Odysseus captured images of the Moon during landing and operated on the lunar surface for 144 hours before entering standby mode.
Prof. Christian and SEAL have more projects on the horizon to develop new technologies for exploring our Moon, other planets, asteroids, and the solar system. These technologies will enable future scientific missions to safely explore challenging destinations and answer scientific questions that were impossible with yesterday’s technology.
SEAL, led by Professor John Christian, collaborated with Intuitive Machines to develop algorithms to guide Athena to the Shackleton crater: a region known for its limited sunlight and cold temperatures. Research Engineer Ava Thrasher (AE 2022, M.S. AE 2024) led Georgia Tech's SEAL team on developing the algorithms used for Athena's flight software.
]]>6:30 p.m. EST launch coverage begins
7:02-7:34 p.m. EST launch window
Stream on NASA+
]]>Intuitive Machines' IM-2 mission lunar lander, Athena, in the company's Lunar Production and Operations Center. Credit: Intuitive Machines
Current training methods limit robots from being produced at the necessary scale to put a robot in every home, said Simar Kareer, a Ph.D. student in the School of Interactive Computing.
“Traditionally, collecting data for robotics means creating demonstration data,” Kareer said. “You operate the robot’s joints with a controller to move it and achieve the task you want, and you do this hundreds of times while recording sensor data, then train your models. This is slow and difficult. The only way to break that cycle is to detach the data collection from the robot itself.”
[VIDEO: Meta Shares EgoMimic Case Study Video]
Other fields, such as computer vision and natural language processing (NLP), already leverage training data passively culled from the internet to create powerful generative AI and large-language models (LLMs).
Many roboticists, however, have shifted toward interventions that allow individual users to teach their robots how to perform tasks. Kareer believes a similar source of passive data can be established to enable practical generalized training that scales the production of humanoid robots.
This is why Kareer collaborated with School of IC Assistant Professor Danfei Xu and his Robot Learning and Reasoning Lab to develop EgoMimic, an algorithmic framework that leverages data from egocentric videos.
Meta’s Ego4D dataset inspired Kareer’s project. The benchmark dataset, released in 2023, consists of first-person videos of humans performing daily activities. This open-source data set trains AI models from a first-person human perspective.
“When I looked at Ego4D, I saw a dataset that’s the same as all the large robot datasets we’re trying to collect, except it’s with humans,” Kareer said. “You just wear a pair of glasses, and you go do things. It doesn’t need to come from the robot. It should come from something more scalable and passively generated, which is us.”
Kareer acquired a pair of Meta’s Project Aria research glasses, which contain a rich sensor suite and can record video from a first-person perspective through external RGB and SLAM cameras.
Kareer recorded himself folding a shirt while wearing the glasses and repeated the process. He did the same with other tasks such as placing a toy in a bowl and groceries into a bag. Then, he constructed a humanoid robot with pincers for hands and attached the glasses to the top to mimic a first-person viewpoint.
The robot performed each task repeatedly for two hours. Kareer said building a traditional training algorithm would take days of teleoperating and recording robot sensory data. For his project, he only needed to gather a baseline of sensory data to ensure performance improvement.
Kareer bridged the gap between the two training sets with the EgoMimic algorithm. The robot’s task performance rating increased by as much as 400% among various tasks with just 90 minutes of recorded footage. It also showed the ability to perform these tasks in unseen environments.
If enough people wear Aria glasses or other smart glasses while performing daily tasks, it can create the passive data bank needed to train robots on a massive scale.
This type of data collection can enable nearly endless possibilities for roboticists to help humans achieve more in their everyday lives. Humanoid robots can be produced and trained at an industrial level and be able to perform tasks the same way humans do.
“This work is most applicable to jobs that you can get a humanoid robot to do,” Kareer said. “In whatever industry we are allowed to collect egocentric data, we can develop humanoid robots.”
Kareer will present his paper on EgoMimic at the 2025 IEEE Engineers’ International Conference on Robotics and Automation (ICRA), which will take place from May 19 to 23 in Atlanta. The paper was co-authored by Xu and School of IC Assistant Professor Judy Hoffman, fellow Tech students Dhruv Patel, Ryan Punamiya, Pranay Mathur, and Shuo Cheng, and Chen Wang, a Ph.D. student at Stanford.
]]>Georgia Tech College of Computing
]]>However, this new generation of self-thinking robots would need security protocols to ensure they aren’t susceptible to hackers. To integrate such robots into society, they must come with assurances that they will behave safely around humans.
It begs the question: can you guarantee the safety of something that doesn’t exist yet? It’s something Assistant Professor Glen Chou hopes to accomplish by developing algorithms that will enable autonomous systems to learn and adapt while acting with safety and security assurances.
He plans to launch research initiatives, in collaboration with the School of Cybersecurity and Privacy and the Daniel Guggenheim School of Aerospace Engineering, to secure this new technological frontier as it develops.
“To operate in uncertain real-world environments, robots and other autonomous systems need to leverage and adapt a complex network of perception and control algorithms to turn sensor data into actions,” he said. “To obtain realistic assurances, we must do a joint safety and security analysis on these sensors and algorithms simultaneously, rather than one at a time.”
This end-to-end method would proactively look for flaws in the robot’s systems rather than wait for them to be exploited. This would lead to intrinsically robust robotic systems that can recover from failures.
Chou said this research will be useful in other domains, including advanced space exploration. If a space rover is sent to one of Saturn’s moons, for example, it needs to be able to act and think independently of scientists on Earth.
Aside from fighting fires and exploring space, this technology could perform maintenance in nuclear reactors, automatically maintain the power grid, and make autonomous surgery safer. It could also bring assistive robots into the home, enabling higher standards of care.
This is a challenging domain where safety, security, and privacy concerns are paramount due to frequent, close contact with humans.
This will start in the newly established Trustworthy Robotics Lab at Georgia Tech, which Chou directs. He and his Ph.D. students will design principled algorithms that enable general-purpose robots and autonomous systems to operate capably, safely, and securely with humans while remaining resilient to real-world failures and uncertainty.
Chou earned dual bachelor’s degrees in electrical engineering and computer sciences as well as mechanical engineering from University of California Berkeley in 2017, a master’s and Ph.D. in electrical and computer engineering from the University of Michigan in 2019 and 2022, respectively. He was a postdoc at MIT Computer Science & Artificial Intelligence Laboratory prior to joining Georgia Tech in November 2024. He is a recipient of the National Defense Science and Engineering Graduate fellowship program, NSF Graduate Research fellowships, and was named a Robotics: Science and Systems Pioneer in 2022.
]]>John (JP) Popham
Communications Officer II
College of Computing | School of Cybersecurity and Privacy
Most hepatic diseases are chronic conditions that will be present over the life of the patient, but early detection improves overall health and the ability to manage specific conditions over time. Additionally, assessing patients over time allows for effective treatments to be adjusted as necessary. The standard protocol for diagnosis, as well as follow-up tissue assessment, is a biopsy after the return of an abnormal blood test, but biopsies are time-consuming and pose risks for the patient. Several non-invasive imaging techniques have been developed to assess the stiffness of liver tissue, an indication of scarring, including magnetic resonance elastography (MRE).
MRE combines elements of ultrasound and MRI imaging to create a visual map showing gradients of stiffness throughout the liver and is increasingly used to diagnose hepatic issues. MRE exams, however, can fail for many reasons, including patient motion, patient physiology, imaging issues, and mechanical issues such as improper wave generation or propagation in the liver. Determining the success of MRE exams depends on visual inspection of technologists and radiologists. With increasing work demands and workforce shortages, providing an accurate, automated way to classify image quality will create a streamlined approach and reduce the need for repeat scans.
Professor Jun Ueda in the George W. Woodruff School of Mechanical Engineering and robotics Ph.D. student Heriberto Nieves, working with a team from the Icahn School of Medicine at Mount Sinai, have successfully applied deep learning techniques for accurate, automated quality control image assessment. The research, “Deep Learning-Enabled Automated Quality Control for Liver MR Elastography: Initial Results,” was published in the Journal of Magnetic Resonance Imaging.
Using five deep learning training models, an accuracy of 92% was achieved by the best-performing ensemble on retrospective MRE images of patients with varied liver stiffnesses. The team also achieved a return of the analyzed data within seconds. The rapidity of image quality return allows the technician to focus on adjusting hardware or patient orientation for re-scan in a single session, rather than requiring patients to return for costly and timely re-scans due to low-quality initial images.
This new research is a step toward streamlining the review pipeline for MRE using deep learning techniques, which have remained unexplored compared to other medical imaging modalities. The research also provides a helpful baseline for future avenues of inquiry, such as assessing the health of the spleen or kidneys. It may also be applied to automation for image quality control for monitoring non-hepatic conditions, such as breast cancer or muscular dystrophy, in which tissue stiffness is an indicator of initial health and disease progression. Ueda, Nieves, and their team hope to test these models on Siemens Healthineers magnetic resonance scanners within the next year.
Publication
Nieves-Vazquez, H.A., Ozkaya, E., Meinhold, W., Geahchan, A., Bane, O., Ueda, J. and Taouli, B. (2024), Deep Learning-Enabled Automated Quality Control for Liver MR Elastography: Initial Results. J Magn Reson Imaging. https://doi.org/10.1002/jmri.29490
Prior Work
Robotically Precise Diagnostics and Therapeutics for Degenerative Disc Disorder
Related Material
Editorial for “Deep Learning-Enabled Automated Quality Control for Liver MR Elastography: Initial Results”
Research Communications Program Manager |
Topic Expertise: Robotics, Data Sciences, Semiconductor Design & Fab |
]]>Since humans often associate objects in a home with the room they are in, Naoki Yokoyama thinks robots that navigate human environments to perform assistive tasks should mimic that reasoning.
Roboticists have employed natural language models to help robots mimic human reasoning over the past few years. However, Yokoyama, a Ph.D. student in robotics, said these models create a “bottleneck” that prevents agents from picking up on visual cues such as room type, size, décor, and lighting.
Yokoyama presented a new framework for semantic reasoning at the Institute of Electrical and Electronic Engineers (IEEE) International Conference on Robotics and Automation (ICRA) last month in Yokohama, Japan. ICRA is the world’s largest robotics conference.
Yokoyama earned a best paper award in the Cognitive Robotics category with his Vision-Language Frontier Maps (VLFM) proposal.
Assistant Professor Sehoon Ha and Associate Professor Dhruv Batra from the School of Interactive Computing advised Yokoyama on the paper. Yokoyama authored the paper while interning at the Boston Dynamics’ AI Institute.
“I think the cognitive robotic category represents a significant portion of submissions to ICRA nowadays,” said Yokoyama, whose family is from Japan. “I’m grateful that our work is being recognized among the best in this field.”
Instead of natural language models, Yokoyama used a renowned vision-language model called BLIP-2 and tested it on a Boston Dynamics “Spot” robot in home and office environments.
“We rely on models that have been trained on vast amounts of data collected from the web,” Yokoyama said. “That allows us to use models with common sense reasoning and world knowledge. It’s not limited to a typical robot learning environment.”
BLIP-2 matches images to text by assigning a score that evaluates how well the user input text describes the content of an image. The model removes the need for the robot to use object detectors and language models.
Instead, the robot uses BLIP-2 to extract semantic values from RGB images with a text prompt that includes the target object.
BLIP-2 then teaches the robot to recognize the room type, distinguishing the living room from the bathroom and the kitchen. The robot learns to associate certain objects with specific rooms where it will likely find them.
From here, the robot creates a value map to determine the most likely locations for a target object, Yokoyama said.
Yokoyama said this is a step forward for intelligent home assistive robots, enabling users to find objects — like missing keys — in their homes without knowing an item’s location.
“If you’re looking for a pair of scissors, the robot can automatically figure out it should head to the kitchen or the office,” he said. “Even if the scissors are in an unusual place, it uses semantic reasoning to work through each room from most probable location to least likely.”
He added that the benefit of using a VLM instead of an object detector is that the robot will include visual cues in its reasoning.
“You can look at a room in an apartment, and there are so many things an object detector wouldn’t tell you about that room that would be informative,” he said. “You don’t want to limit yourself to a textual description or a list of object classes because you’re missing many semantic visual cues.”
While other VLMs exist, Yokoyama chose BLIP-2 because the model:
Yokoyama also tested the Spot robot to navigate a more challenging office environment. Office spaces tend to be more homogenous and harder to distinguish from one another than rooms in a home.
“We showed a few cases in which the robot will still work,” Yokoyama said. “We tell it to find a microwave, and it searches for the kitchen. We tell it to find a potted plant, and it moves toward an area with windows because, based on what it knows from BLIP-2, that’s the most likely place to find the plant.”
Yokoyama said as VLM models continue to improve, so will robot navigation. The increase in the number of VLM models has caused robot navigation to steer away from traditional physical simulations.
“It shows how important it is to keep an eye on the work being done in computer vision and natural language processing for getting robots to perform tasks more efficiently,” he said. “The current research direction in robot learning is moving toward more intelligent and higher-level reasoning. These foundation models are going to play a key role in that.”
Top photo by Kevin Beasley/College of Computing.
]]>Yokoyama presented a new framework for semantic reasoning at the Institute of Electrical and Electronic Engineers (IEEE) International Conference on Robotics and Automation (ICRA) last month in Yokohama, Japan. ICRA is the world’s largest robotics conference.
Yokoyama earned a best paper award in the Cognitive Robotics category with his Vision-Language Frontier Maps (VLFM) proposal.
]]>Communications Officer
School of Interactive Computing
]]>Georgia Tech’s Simon Sponberg recently collaborated with researchers at the University of Washington, Simon Fraser University, University of Colorado Boulder, and Stanford Research Institute to answer one deceptively complex question: Why can’t robots outrun animals?
“This work is about trying to understand how, despite have some really amazing robots, there still seems to be a gulf between the capabilities of animal movement and what we can engineer,” says Sponberg, who is Dunn Family Associate Professor in the School of Physics and School of Biological Sciences.
Recently published in Science Robotics, their study systematically examines a suite of biological and robotic runners to figure out how to further advance our best robotic designs.
“In robotics design we are often very component focused — we are used to having to establish specifications for the parts that we need and then finding the best component solution,” said Sponberg, who also serves on the executive committee for Georgia Tech's Neuro Next Initiative. “This is of course not how evolution works. We wondered if we systematically analyzed the performance of animals in the same component way that we design robots, if we might see an obvious gap.”
The gap turns out not to be in the function of individual robotic components, but rather the ability of those components to work together in the seamless way biological components do, highlighting a field of opportunity for new research in robotic development.
“This means that the frontier is not necessarily figuring out how to design better motors or sensors or controllers,” says Sponberg, “but rather how to integrate them together — this is where biology really excels.”