Students will work with faculty mentors from both institutions and complete advanced coursework and a capstone project focused on developing and applying technologies in healthcare. The master’s degree is offered through Georgia Tech’s interdisciplinary Master of Science in Robotics program, which brings together faculty and expertise from six schools, including the Wallace H. Coulter Department of Biomedical Engineering.
According to Jaydev Desai, professor and Cardiovascular Biomedical Engineering Distinguished Chair in the Wallace H. Coulter Department of Biomedical Engineering, the pathway will help future physicians develop the technical expertise to understand and apply emerging technologies as they shape patient care.
Read the full story on the Emory School of Medicine website.
]]>"The most meaningful breakthroughs happen when intelligent hardware and thoughtful software come together to genuinely help people,” said Tapas Roy, vice president of Device Software and Services at Amazon. “Georgia Tech's world-class expertise in robotics and engineering, paired with Amazon's work in on-device intelligence and artificial intelligence, will accelerate our understanding of systems that can both respond to prompts and anticipate customer needs. I'm excited to see how this collaboration advances the science of how humans and intelligent machines work together.”
The Hub’s initial focus areas will explore challenges in autonomous systems, next-generation logistics infrastructure, and low-Earth-orbit communications, drawing on Georgia Tech’s well-known interdisciplinary research approach.
Georgia Tech is also joining the Fauna Robotics' ecosystem, using the Sprout research platform to study how humans and robots can live, work, and play side by side. This collaboration opens new ways to study mobility, perception, manipulation, and human-robot interaction. Leadership and alumni recently celebrated the launch at Lab 126, with guest appearances from Sprout and Buzz, the Georgia Tech mascot.
The Hub will support:
• Joint research projects between Georgia Tech faculty and Amazon scientists across scientific disciplines
• Regular symposia and workshops to foster collaboration between researchers from both institutions and the broader Atlanta tech community
• Community outreach programs designed to broaden participation from interdisciplinary scholars and innovators
“Georgia Tech and Amazon have worked together for years across research, technology, and student development," said Tim Lieuwen, Georgia Tech executive vice president for Research. "This new hub puts our partnership to work for Georgia. It connects research, education, and industry in ways that will strengthen our state’s innovation economy and create new opportunities for our students.”
About Georgia Tech
Georgia Institute of Technology is one of the nation’s top public research universities, preparing leaders who advance technology and improve the human condition. The Institute serves more than 56,000 students from all 50 states and over 146 countries and drives innovation and economic growth through research that benefits Georgia, the nation, and the world.
About Amazon in Georgia
Since 2010, Amazon has invested more than $30 billion in Georgia, supporting more than 30,000 full- and part-time jobs. Amazon’s research and technology teams in the Atlanta area work on challenges spanning supply chain optimization, logistics technology, and cloud computing. The establishment of the Georgia Tech–Amazon Science Hub further deepens Amazon’s engagement with one of the nation’s leading research institutions.
About Fauna Robotics
Fauna Robotics builds Sprout, a development platform designed to put advanced robotics capabilities into the hands of researchers and developers. Sprout gives builders hands-on access to study mobility, perception, motor control, and manipulation with the ambition to deploy robots into human-centric spaces.
To improve robot performance and advance memory capabilities needed for reliable home use, Georgia Tech researchers have developed a new benchmark to evaluate the memory capabilities of vision-language models (VLMs).
That includes Gemini 2.0-flash and GPT-4o.
In-home assistive robots could clean and organize a home while the owner is away, but without enhanced memory capabilities, they could cause more problems than they solve.
They could, for example, cause a major inconvenience if they move important objects like wallets and keys but are unable to tell the owner where they put them.
A new benchmark created by Georgia Tech researchers tests the memory capability of vision-language models (VLMs), including Gemini 2.0-flash and GPT-4o, to determine whether they could improve robot performance.
Zsolt Kira, an associate professor in Tech’s School of Interactive Computing, said that most active VLMs can process only a few hundred images at a time before forgetting them.
“There wasn’t a benchmark that could test these specialized capabilities that we might want from a robot,” Kira said. “Our purpose in creating one is to spur research in this area so that others can develop methods to solve this problem.”
The team’s benchmark consists of memory evaluations based on 60 tasks that require continuous engagement and contextual and environmental awareness. Three task categories — spatial, temporal, and multi-goal — are highly difficult.
“You can chat with a language model like ChatGPT, and it remembers all the text from your previous conversations with it because the text data is small in terms of storage,” Kira said.
“Embodied agents and robots observe a lot of videos, and those tend to be a lot harder to recall. If you ask it where your diary is, it must efficiently store images of your diary and where it is.
“That’s just a few frames out of a million that it recorded since yesterday, and it must know where that is relative to the spatial layout of the house.”
Karmesh Yadav, a Ph.D. student in Kira’s lab, said users don’t want to repeat information about their preferences after deployment. They also expect the robot to maintain long-term memory.
“In reality, you would want it to have a memory of the entire time it has been in your house, whether that’s days or years,” Yadav said.
The team’s benchmark evaluates memory performance over an average one-day period. It challenges the models with commands like “navigate to an object you did not interact with yesterday” or “navigate to a room you did not visit yesterday.”
The results show that current state-of-the-art models have a long way to go before they can be used in embodied agents. The best model achieved roughly a 50% success rate on high-level difficulty tasks.
Surprisingly, this model wasn’t Gemini or GPT VLMs. It was an open-source reasoning model from Qwen.
“If you use an open-source reasoning model, even one that’s small in size, it can match the performance of closed-source non-reasoning models,” Yadav said. “Reasoning models can look at the video and translate it into text. It describes what’s going on in the video in words and determines the right frame to choose in its final output.”
Kira said that converting images into descriptive text that language models easily understand could be one way to circumvent the memory storage dilemma. Another way could be to assign a hierarchy that teaches the model to identify irrelevant or redundant images and erase them from storage.
Whatever future progress researchers make in those areas, Kira and his students are confident that their benchmark will remain relevant.
“This benchmark is scalable,” said Ph.D. student Yusuf Ali. “If two years from now, videos are easy for a model to understand and store, you can use the same infrastructure we propose and scale up the problem’s difficulty.”
Yadav and Ali are co-first authors of a paper on the benchmark, presented in early September at the 2026 European Conference on Computer Vision in Sweden.
For more information about the project, click here.
]]>“It was something I had wanted since I was a kid,” he said. “Growing up in India, it wasn't known for martial arts.”
Agrawal knew he needed something to help him balance his physical health with the challenges of graduate school. He checked the list of classes offered by Georgia Tech’s Campus Recreation. One stood out: Georgia Tech Taido.
Taido is a niche martial art that applies a modern, scientific approach to traditional Okinawan karate. There is only one school in the U.S. that teaches it — Taido Karate in Peachtree Corners, Ga., which is currently the world’s largest taido school.
Taido means “the way of the body and the mind.” It was invented in the 1960s by Seiken Shukumine.
One of Shukumine’s students, Mitsunobu Uchida, brought the new martial art to the U.S. when he immigrated in 1975. Uchida decided that Atlanta would be an ideal place to open a new school.
Since then, generations of students have trained at Taido Karate, including Georgia Tech alumnus Bryan Sparks (EE 2000), and Andy Fossett, who started as a student at Tech in 1996 before transferring to Georgia State.
Sparks and Fossett learned taido together as kids. Both became black belts before graduating high school.
Shortly after becoming students at Tech, they started an official taido club on campus. They taught their first class in 1996, which only had one student. By the next semester, they had almost a dozen.
“The class was held in an open area, and a lot of people would walk by and see us training,” Sparks said. “That was the best advertisement we could ever have. We’d invite them to work out, and it grew from there. We did some demonstrations around campus in the early days.”
The taido club is still going strong and celebrating its 30th anniversary this year.
Agrawal discovered it in 2018 as a first-year Ph.D. student. He earned his doctorate in robotics from Georgia Tech’s School of Interactive Computing in 2025.
While keeping his Ph.D. studies first, Agrawal trained for five years to earn a black belt. He said the process was the most physically and mentally challenging he’s ever experienced.
“The tenets of taido teach you how to compose yourself and how to have a good mindset, which becomes helpful in dealing with daily life stress,” Agrawal said. “It helps you focus on what’s important in the moment.”
“There were times when I felt like nothing was going right, and I kept asking ‘What am I even doing?’ But I knew if I could do some of the harder things in taido, there’s no reason I couldn’t do the same thing in my Ph.D.”
Agrawal’s commitment paid off in 2022 when he became one of five students from Tech Taido to pass the black belt test that year.
“The test is grueling,” he said. “It’s four hours long. It’s the hardest thing I’ve ever done physically, and I’ve run several marathons. They test you on everything you’ve learned from white belt level to brown belt level. I was training five days a week for five months.
“It was one of the best days of my life. It felt like no matter what happened, I could move forward with my life and be proud of that accomplishment.”
Agrawal landed a remote job with a robotics startup after graduating. His new position allowed him to stay in Atlanta and continue with Georgia Tech Taido as an instructor.
He said the unique aspect of taido is its emphasis on community and culture, which can be attributed to Uchida’s influence.
“A lot of times, American senseis open dojos after learning martial arts overseas. They bring the techniques and the discipline over, but they don’t bring the culture that comes with it. I love how taido is a part of our lives. It’s not a separate workout; it’s a communal thing.”
Agrawal teaches classes with Lacy Hardcastle, who took over as lead instructor when Sparks moved out of the area in 2019. Hardcastle took her first Taido class at Tech in 2007.
Hardcastle said students trying taido for the first time will get a challenging workout.
“I didn’t know how weak my legs were until I took taido,” she said. “It kicked my butt the first semester. We do whole body conditioning with lots of push-ups, lunges, squats and other exercises. It’s a very athletic martial art and requires whole-body fitness and control.”
Hardcastle describes the environment as welcoming and supportive, with regular get-togethers outside of class and a biennial trip to Japan with the Taido Karate school.
“We have a great social community where we support each other,” she said. “We occasionally go out to social events together. Sometimes we’ll get together to watch MMA fights, things like that. But we also work out hard.”
Georgia Tech Taido classes are held every Monday and Wednesday from 8:30 p.m. to 9:45 p.m. in Studio A of the Georgia Tech Campus Recreation Center. The registration cost for the fall semester is $75. To register, visit https://crc.gatech.edu/martial-arts-4/.
]]>It began with a focus on helping computer systems incorporate human language into their perception of the world. Could a robot tell humans that a heavy object is straining its arm? Does it know that a container makes a sound when it is picked up or dropped?
Now, Thomason’s research has become more human-centered, which is one reason he feels at home as he joins the faculty at Georgia Tech’s School of Interactive Computing as an associate professor.
Thomason comes to Georgia Tech after spending five years as an assistant professor at the University of Southern California’s Thomas Lord Department of Computer Science.
He will move his Grounding Language in Actions, Multimodal Observation, and Robots (GLAMOR) lab to Tech. The lab explores methods for connecting natural language processing to robotics.
While pursuing his computer science Ph.D. at the University of Texas, Thomason researched early NLP systems before large-language models such as ChatGPT and Google Gemini were introduced.
Thomason then expanded into robotics to explore how roboticists can create new ways to communicate with robots using computational representations of language.
“The idea was you could do something that looked like robot programming without as much engineering work if you used one of these coding modules to create a language-code interface,” Thomason said.
When OpenAI and Google released their commercial LLMs in 2022, Thomason felt a seismic shift.
“That area of NLP and robotics became more crowded and accessible with the advent of (LLMs),” Thomason said. “It created a huge inroad to roboticists being able to plug and play language technologies in a way they couldn’t before. Robotics hardware has also become much cheaper and more standardized.”
But as LLMs became universal tools accessible to everyone, Thomason said he started thinking more about how he could improve optimization for users.
“I became most interested in how language interfaces, starting in 2022, were suddenly everywhere,” he said. “Everybody had a chatbot on their phone. I wanted to see what we could do as researchers to make the interaction between an AI system like a language model and a person smoother and more beneficial.”
Thomason said he felt he needed a new environment where end users were at the center of the technology design process. He was drawn to the School of Interactive Computing for its reputation in human-computer interaction research.
“That was a big factor in my decision to move to the School of Interactive Computing, where you value that kind of human-centered computing,” he said.
“With the strong HCI researchers here, I have an opportunity to learn more from my colleagues about how to run high-quality human-subject studies, conduct participatory design, and identify actual users of a potential system and talk to them about what would be helpful.”
]]>Karamcheti said there are times when he doesn’t see any humans at all. Those instances reinforce the perception that robots are replacing humans rather than assisting them.
“You have humanoid robots doing things like folding laundry, and you see videos of it on social media,” Karamcheti said. “It bothers me that in all these videos, I see the robot doing these tasks alone without a single person around them. True human-robot collaboration is missing.”
As he joins the Georgia Tech School of Interactive Computing as an assistant professor, Karamcheti seeks to improve human-robot collaboration and maximize output through his research.
Karamcheti comes to Tech after earning his Ph.D. in computer science (CS) from Stanford University. He also holds bachelor’s degrees in CS and literary arts from Brown University.
In addition to human-robot interaction, he has conducted research on fine-tuning vision-language-action (VLA) models.
VLAs are artificial intelligence (AI) systems typically built by fine-tuning vision-language models (VLMs) or large language models (LLMs). They use visual perception, natural language, and physical action to perform tasks.
“I have a high-level vision of what I think robotics should be, but there are many winding paths I like to take along the way as I try to understand why something works the way it does,” Karamcheti said.
Karamcheti said his vision was inspired by the movie Iron Man and by how the main character, Tony Stark, interacts with robots in his lab.
“I don’t want a robot that replaces all the work I’m doing in my life,” he said. “I want a robot that’s embedded in my life like my laptop or smartphone.
“They’re making it possible for me to do more things. I also want to make it so that robots start doing more and learning more when I leave the room.”
Karamcheti said the School of IC is a “dream” job for him because of its renowned faculty in human-robot interaction. It’s the ideal place to keep humans at the center of his research.
“We must design models around interaction and the idea that there’s going to be a person in the room. Robots don’t need to be perfect at any one thing. They just need to be able to ask for help when it’s needed and learn from those teaching interactions.”
Karamecheti said many robotics researchers are pursuing robots that are as close to perfect as possible so they can operate without human oversight. However, the amount of money and data needed to get there is substantial.
“I think you see diminishing returns with scale. If a robot performs tasks correctly 90% of the time, the data required to go from 90% to 95% is much less than that required to go from 95% to 99%. Many robotics companies and labs are investing to reach a 99% success rate,” he said.
Karamcheti added that the pursuit of perfection shouldn’t prevent a robot from being deployed. They could be deployed at 95% and increase their success rate by working with and learning from humans.
“Once I have a robot that’s 95% capable, I understand that 5% of failures will occur for a specific task. If I know a human is always going to be near the robot, I can design a system that sequences tasks and anticipates human intervention.”
Karamcheti is recruiting students to join his lab who bring new perspectives to robotics.
“I’m looking for students who aren’t afraid to challenge common assumptions and who enjoy hacking and building,” he said. “I want to work with students who have strong opinions about how robotics should look.”
]]>Karamcheti comes to Tech after earning his Ph.D. in computer science (CS) from Stanford University. He also holds bachelor’s degrees in CS and literary arts from Brown University.
In addition to human-robot interaction, he has conducted research on fine-tuning vision-language-action (VLA) models.
]]>Dunkin is an IISE Fellow, the highest classification of IISE membership that recognizes outstanding leaders of the profession who have made significant, nationally recognized contributions to industrial and systems engineering. She has served as chief information officer at the U.S. Department of Energy and the U.S. Environmental Protection Agency, as well as chief strategy and innovation officer at Dell Technologies. She has also held numerous leadership positions at Hewlett Packard, largely in the areas of manufacturing, research and development and IT. She currently serves as a distinguished external fellow with Georgia Tech’s Strategic Energy Institute.
Dunkin holds appointments as a distinguished professor of the practice in Georgia Tech’s School of Cybersecurity and Privacy and in the Jimmy and Rosalynn Carter School of Public Policy. She is also a distinguished external fellow at the Georgia Tech Research Institute (GTRI).
“The Emerging Technologies forum is a great opportunity for practicing ISEs to get exposure to a range of emerging technologies in a short period of time and identify those that they want to experiment with and implement in their organizations,” Dunkin said.
The Aug. 26 event, which will be held virtually from 1-5 p.m. EDT, will explore technologies shaping the future of industrial and systems engineering, including artificial intelligence, digital twins, and robotics and autonomy.
As event chair, Dunkin will be joined by experts from the healthcare, aerospace, logistics, manufacturing and technology industries, including representatives from Royal Caribbean, The Boeing Company, UPS and other organizations.
The complimentary event is open to engineers and improvement professionals of all backgrounds. Registration is available through the IISE Emerging Technologies Forum website.
For most, that heart-pounding uncertainty ends the moment the foot finds solid ground. But for many individuals living with conditions like stroke or spinal cord injury (SCI), that sense of disconnect is a permanent reality.
“These conditions of course have a huge effect on our ability to move around and be independent — but the other side of it is the sensory feedback that we lose,” says Matthew Flavin, an assistant professor in the School of Electrical and Computer Engineering. Most rehabilitation treatments primarily focus on restoring movement, but “even if you have motor control, if you can’t feel when your foot's touching the ground it can be really hard for you to move around safely.”
In a new study published in Proceedings of the National Academy of Sciences, Flavin and an interdisciplinary team of researchers introduce a way to bridge this gap: a wearable “sensory substitution” system that translates foot pressure into high-tech patterns of heat and vibration they can feel elsewhere.
The system uses high-resolution pressure-sensing insoles designed by the team, which are placed inside a user's shoes to record how their weight shifts in real-time. This data is streamed via Bluetooth to a flexible, skin-conformable array of haptic receivers worn on the forearms, a part of the body that often retains sensation in SCI. The receivers give quick pressure feedback through vibration, while also alerting the user to longer-term pressure “hotspots” through heat.
“One of the limitations of a lot of approaches in haptics is that you're having to map a missing sense onto a completely different sense,” says Flavin. “We’re keeping the type of information that we're missing, which is the distribution of pressure, and we're just basically putting it on a different part of their body.”
Rerouting the lost sensation was key to making the device intuitive to learn. Participants were able to correctly identify the “feel” of the ground through their arms with high accuracy within a mere two-hour session. When tested with a small group of participants with stroke or SCI, the wearable significantly improved standing balance and led to steadier walking.
“What’s encouraging about these early results is that participants appeared to use the feedback in ways that supported balance and walking,” says John Rogers, a materials science and engineering professor at Northwestern University who collaborated on this study. “Our study suggests that providing pressure information through another part of the body could be a practical path for helping people compensate for lost sensation.”
While vibration provides immediate feedback for walking and balance, the team views the thermal feedback as a tool for long-term health. Heat is a slower, low-frequency signal that could alert patients to pressure hotspots, potentially preventing diabetic foot ulcers or pressure injuries for those who are bedridden or use wheelchairs.
The small, lightweight system is completely untethered, making it suitable for use during daily activities in and outside the clinic. It’s also highly adaptable to different injury types, which is ideal for conditions as variable as stroke, SCI, and diabetic neuropathy. Placement of the haptic receivers can be adjusted based on where a patient has the most sensation, and the sensitivity of the insoles can be tailored to each patient.
As a member of several of Georgia Tech’s Interdisciplinary Research Institutes — the Institute for Neuroscience, Neurotechnology, and Society, the Institute for Robotics and Intelligent Machines, and the Parker H. Petit Institute for Bioengineering and Biosciences — Flavin credits the project’s success to an interdisciplinary effort and deep engagement with clinicians and patients.
“This reinforces the importance of really engaging with your stakeholders very early on,” says Flavin. “If you're not continually refining that concept with those stakeholders, you quickly find that they might be looking for something that your device isn't delivering.”
With new funding from the National Science Foundation (NSF), the team is now working to make the technology even smaller and more reconfigurable, moving closer to a standard wearable for daily clinical use.
DOI: https://doi.org/10.1073/pnas.2536577123
]]>Photos:
Maxwell Guberman
RAI calls the bicycle robot an ultra-mobility vehicle (UMV). It can reach a height of 3 feet and can jump from the floor onto a platform.
The contributions of a Georgia Tech Ph.D. student helped make these feats possible through a robot control policy he developed.
Jeonghwan Kim, who is pursuing a Ph.D. in robotics under the advisement of Associate Professor Sehoon Ha, spent two semesters interning at RAI. His task was to design a policy to teach the UMV to land after a flip.
The result was iterative motion imitation (IMI), a novel method that imitates flip trajectories generated from prior examples. Kim said the robot bases its flip on a demonstration, and human engineers reconstruct and refine the flip path through simulation to fill in the gaps.
“To guide the robot to flip, we started with an imperfect trajectory generated by a motor-based controller and then ran simulations,” Kim said. “It’s an unstable trajectory, but we use it as a guide to train a single policy that can track it as it lands and tries to balance itself.”
Kim interned under the supervision of Shamel Fahmi, a research scientist at the RAI Institute. RAI has been developing the UMV for nearly three years.
“We wanted to work on a different robot morphology that wasn’t legs or legs with wheels,” Fahmi said. “That’s when we thought of working with bikes.
“We want to merge the athleticism of (Boston Dynamics’) Atlas with the mobility of a bike. We wanted a robot that can go anywhere, do parkour, and acrobatics.”
Fahmi said that before Kim arrived, the research team had trouble getting the UMV to land consistently without breaking or falling.
The UMV has two joints — an upper and a lower. The upper joint contains the motors and pulls the lower joint along as it propels into the air. The problem is getting the lighter lower joint to absorb the impact of landing without being crushed by the heavier upper joint.
“That’s what brings reinforcement learning into the equation,” Fahmi said. “We teach the robot to minimize its impact on the ground to land gracefully.”
Fahmi said that Kim proved the imitation examples the robot learns from don’t have to be perfect. The process takes some time, but all it needs is a rough idea to get started.
“You can have an imperfect sketch and then constantly refine it,” Fahmi said. “The first time, it’s not going to go well.
“We don’t care about torque or power limits as long as it does the motion. Then we’ll have a slightly better reference, repeat it, and imitate it again. In every iteration, we can add more parameters.”
Kim said he felt the pressure of time constraints during his two semesters with RAI as he worked to achieve consistent, successful landings. Even though he had multiple UMVs to experiment with, they broke down dozens of times. Each time one broke, a hardware team at RAI had to repair it.
“There was a lot of pressure to not only get this working before my internship ended, but also knowing there are costs behind every failed attempt, and every time the robot breaks, it takes time to repair it,” Kim said.
“It took almost five months for it to land without breaking. Then we needed two more months for it to stay balanced after the landing. It requires a lot of engineering effort to achieve a robust control policy for a safe flip.”
By the time Kim left RAI, the IMI policy had achieved consistent, seamless landings.
“The jump right now is what we call the visitor demo,” Fahmi said. “If there are guests coming over to see it, we want to show them something that is extremely impressive, but also, more importantly, extremely reliable. It never fails.
“It was only possible because of the huge effort we put into designing, maintaining, and continuously improving the robot.”
Kim authored a paper on his framework and will present it at this week’s International Conference on Robotics and Automation (ICRA) in Vienna.
For more information about the UMV project, please visit the RAI blog or watch their video on YouTube.
]]>Jeonghwan Kim, who is pursuing a Ph.D. in robotics under the advisement of Associate Professor Sehoon Ha, spent two semesters interning at RAI. His task was to design a policy to teach the UMV to land after a flip.
]]>The new technology is also set to revolutionize the scale of policy training data collection, which is essential to advancing robotic capabilities and meeting growing production demand.
COBALT is a mobile app that turns smartphones into controllers for robot arms. With a secure Wi-Fi connection to a server, users can move their phones in any direction, and the robot arm will mirror the motion — from anywhere in the world.
Ayush Agarwal, a Ph.D. student in Georgia Tech’s School of Interactive Computing who leads a research team developing COBALT, said it works like the games people play on smartphones. Users can press a button to have the arm grasp an object, move it, and release it with another button.
Agarwal conducted several user studies with participants in nine countries who remotely operated robot arms inside Georgia Tech’s People, AI & Robotics (PAIR) Lab. The lab is directed by Assistant Professor Animesh Garg, who advises Agarwal.
“We built an entire distribution system for remote teleoperation scaled to where we had people from Indonesia, India, and Pakistan operating for us,” Agarwal said. “They were novice operators who had never done it before. By collecting data from these new users, we showed that we can train policies to automate certain tasks.”
Garg envisions a world where data collection for policy training is done through crowdsourcing. He began working toward this goal 10 years ago as a postdoc at Stanford University, when he developed RoboTurk, an earlier version of COBALT.
“There is a large-scale data collection requirement for mass robot production to be possible, and it will not be solved purely through simulation,” Garg said.
“Our idea was, what if we could get almost every person on the planet to be a passive source for data collection? There are almost five billion people who have smartphones and know how to use them.”
Another major implication of COBALT could be expanded access to CS and robotics education.
Students can learn to operate a robot remotely in any classroom. In fact, Garg and his lab recently hosted students from Midtown High School in Atlanta to demonstrate COBALT and let them control robot arms from a phone.
Garg also sees the possibility of a “gig economy” in which people pay remote operators to control assistive robots in their homes and complete household chores for them.
“It could be Uber for robots,” he said. “People who want to log onto the platform can do so at their convenience and for as long as they want.”
Companies with robot-dependent labor tasks could also use the platform to enable human oversight.
“If I deploy a robot in a factory that achieves high autonomy for most tasks, but there are still times it needs help, a human could operate the robot from anywhere in the world,” Garg said.
Agarwal’s studies showed that people prefer to interact with and control a robot using a smartphone rather than virtual reality (VR) headsets, controllers, keyboards, mice, or other devices.
“The phone is a more intuitive interface and can provide data quality that’s on par with other commonly used devices,” he said.
Agarwal also said there is minimal latency in the video feed sent back to operators on the other side of the world. That’s because the amount of data being processed is small.
The data is carried over Web Real-Time Communication (WebRTC), the same technology used by many streaming services and web conferencing platforms such as Zoom and Google Meet.
“There’s a connection from your phone to the teleoperation server, which is connected to the robots,” Agarwal said.
“Then there’s another connection from the teleoperation server back to the user, which allows for a video stream. We need low latency on both because you don’t want the user to move their phone and wait 10 seconds to see the visual feed.”
Agarwal is the co-lead author of a paper on COBALT that is being presented at the IEEE International Conference on Robotics and Automation this week in Vienna. He said the paper stands out because it has moved from theory to the implementation of an entire distribution network.
“The real novelty of our paper is the systems that we build around it to actually support the scaling of remote operation and data collection at a global level,” he said.
]]>