Title: Grasp Contact between Hand and Object: Capture, Analysis, and Applications

Samarth Manoj Brahmbhatt - https://samarth-robo.github.io

Ph.D. Candidate in Robotics

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: April 30, 2020

Time: 1 pm to 3 pm (EST)

Location: https://primetime.bluejeans.com/a2m/live-event/gewfgupj
Remote only because of GT COVID-19 guidelines.

Join through the BlueJeans Events app or web browser (instructions).

Committee

Dr. James Hays (advisor)

Dr. Charles C. Kemp

Dr. C. Karen Liu (Stanford University)

Dr. James M. Rehg

Dr. Yaser Ajmal Sheikh (Carnegie Mellon University, Facebook Reality Labs)

Abstract

Contact is an important but often oversimplified component of human grasping. Capturing hand-object contact in detail can lead to important insights about grasping behavior, and enable applications in diverse fields like virtual reality and human-robot interaction. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. Lack of ground-truth data has significantly influenced research in this field.

We introduce the use of thermal cameras to capture detailed ground-truth hand-object contact (called contact maps), and techniques to simultaneously capture other data modalities like 3D hand pose, object pose, and multi-view RGB-D grasp videos. This has resulted in ContactDB and ContactPose, two large-scale and diverse datasets of participants grasping 3D-printed household objects with functional intents.

Analysis of this data confirms some long held intuitions about hand-object contact, and also reveals some surprising new patterns. We also train machine learning models for diverse contact map prediction from object shape, and for contact modeling from object shape and grasp information.

Next, we present ContactGrasp, an algorithm that uses object shape and a contact map to synthesize functional grasps for kinematically diverse hand models, including robotic end-effectors. Finally, we investigate whether the contact data captured by thermal cameras encodes contact pressure in addition to contact locations. We find that (subject to certain conditions) the structure of our contact data indeed includes information about contact pressure.