event

PhD Defense by Weiyu Liu

Primary tabs

Title: Grounded Semantic Reasoning for Interacting with Real-World Objects

 

Date: Monday, November 14th, 2022

Time: 10:00 AM – 12:00 PM EST

Location: Zoom meeting (https://gatech.zoom.us/j/9702258427)

 

Weiyu Liu

Robotics Ph.D. Student

School of Interactive Computing

Georgia Institute of Technology

 

Committee:

Dr. Sonia Chernova (Advisor) – School of Interactive Computing, Georgia Institute of Technology

Dr. Charlie Kemp – Department of Biomedical Engineering, Georgia Institute of Technology

Dr. Jesse Thomason – Department of Computer Science, University of Southern California

Dr. Animesh Garg – Department of Computer Science, University of Toronto

Dr. Chad Jenkins – Department of Electrical Engineering and Computer Science, University of Michigan

 

Abstract:

To operate in unstructured environments such as homes and offices, robots need to manipulate novel objects while adapting to changes in environments and goals. Semantic knowledge about objects, which represent relations between object categories, locations, properties, and uses, can reveal meaningful connections between problems and environments. However, closely integrating semantic knowledge and sensorimotor data (e.g., object point clouds, 6-DoF poses, and attributes detected with multimodal sensing) remains a key challenge because the two types of data have drastically different characteristics in terms of modality, complexity, and levels of abstraction. This thesis develops semantic reasoning frameworks capable of modeling complex semantic knowledge grounded in robot perception and action. We show that grounded semantic reasoning enables robots to more effectively perceive, model, and manipulate objects in real-world environments. Specifically, this thesis makes the following contributions: 1) a survey of semantic reasoning providing a unified view for the diversity of works in the field, 2) a method for predicting missing relations in large-scale knowledge graphs by leveraging type hierarchies of entities, 3) an n-ary knowledge representation for predicting unknown properties of objects in various environmental contexts, 4) semantic grasping methods that account for a broad sense of contexts and achieve generalization by reasoning about semantics of tasks and objects, 5) methods for rearranging novel objects into semantically meaningful spatial structures based on high-level language instructions.

Status

  • Workflow Status:Published
  • Created By:Tatianna Richardson
  • Created:11/04/2022
  • Modified By:Tatianna Richardson
  • Modified:11/04/2022

Categories

Keywords