Title: Cross-Embodiment Imitation for Robot Whole-body Skill Learning

Date: Tuesday,  April 15th 2025
Time: 12:00 PM -  2:00 PM EST
Location: Klaus 3126, Zoom Link

Tianyu Li
Ph.D. Student in Computer Science
School of Interactive Computing 
Georgia Institute of Technology 
https://easypapersniper.github.io/

 
Committee: 
Dr. Sehoon Ha (Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Danfei Xu – School of Interactive Computing, Georgia Institute of Technology
Dr. Greg Turk – School of Interactive Computing, Georgia Institute of Technology
Dr. Karen Liu – Computer Science Department, Stanford University
Dr. Marco Hutter –  Robotic Systems Lab, ETH Zürich

Abstract: 
Robots need to acquire natural, human-like skills to assist humans with a wide range of tasks. A direct and intuitive approach to learning these skills is through imitation of human demonstrations. However, robots often differ drastically from humans in both morphology and dynamics—with forms ranging from quadrupedal and wheeled robots to multi-arm manipulators. This embodiment gap poses significant challenges for transferring complex, whole-body human motions to robots.

In this thesis, we introduce four frameworks that enable robots to learn sophisticated whole-body skills from human demonstrations. 1) we present Adversarial Correspondence Embedding (ACE), an unsupervised learning framework that maps human motion dataset to motion spaces of morphologically different robots. ACE enables a variety of human motions—such as playing volleyball or punching—to be retargeted to robots with significant different embodiments. 2) CrossLoco, an unsupervised reinforcement learning framework that jointly learns robot control and human-to-robot motion correspondence. Without requiring prior knowledge of robot skills, CrossLoco enables the translation of human motions into executable robot behaviors. 3) We introduce RobotMover, an imitation learning framework that allows robots to manipulate large objects, such as furniture, by imitating human-object interactions. We demonstrate that the learned policy enables robust and versatile manipulation of diverse objects along various target velocity trajectories. 4) we propose Cross-embodiment Interaction Imitation, a framework for learning complex two body interactive behaviors—such as dancing, sparring, and handshaking—from human demonstrations and transferring them to robots with different body structures. In ongoing work, we explore improved techniques for generalizing imitation across arbitrary embodiments ("any-to-any" motion imitation), focusing on universal motion representation.