Abstract: Designing and assembling complex products traditionally demands extensive manual effort and expert knowledge. In this talk, I will introduce Prompt-to-Product, an automated pipeline that transforms natural language prompts into real-world assembly creations. Using LEGO bricks as a versatile platform, we automate both the design and physical construction of intricate assemblies.

We first present BrickGPT, a fine-tuned large language model with enhanced physical reasoning capability, enabling it to reliably generate assembly designs that are both creative and physically buildable. Next, we introduce BrickMatic, a dexterous bimanual manipulation platform that employs a skill-graph-based neural-symbolic approach to generate precise robot trajectories for long-horizon assembly tasks. BrickMatic achieves sub-millimeter accuracy and scales to complex builds with over 100 components.

Together, BrickGPT and BrickMatic bridge the gap between imagination and realization, making it dramatically easier for users to turn creative ideas into physical products. Through comprehensive user studies, we demonstrate that the Prompt-to-Product framework significantly lowers barriers, reduces manual effort, and empowers anyone to design and assemble sophisticated creations.

Bio: Changliu Liu is an associate professor in the Robotics Institute, School of Computer Science, Carnegie Mellon University (CMU), where she leads the Intelligent Control Lab. Prior to joining CMU in 2019, Dr. Liu was a postdoc at Stanford. She received her Ph.D. degree from University of California, Berkeley and her bachelor degree from Tsinghua University. Her research interests lie in the design and verification of human-centered intelligent systems with applications to manufacturing and transportation and on various robot embodiments. Dr. Liu co-founded Instinct Robotics, a robotics company for intelligent manufacturing. Her work has been recognized by NSF Career Award, Amazon Research Award, Ford URP Award, Advanced Robotics for Manufacturing Champion Award, Young Investigator Award at International Symposium of Flexible Automation, IEEE RAS Early Academic Career Award in Robotics and Automation, IFAC Robotics Outstanding Young Researcher Award, and many best/outstanding paper awards.