Title: Physics-Based Reinforcement Learning for Autonomous Manipulation

Jonathan Scholz

Robotics PhD Candidate

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: Monday, August 17th, 2015

Time: 8:30am-11:30pm EST

Location: Klaus Advanced Computing Building (KACB) 1116E

Committee:

--------------

Dr. Charles Isbell  (School of Interactive Computing, Georgia Institute of Technology)

Dr. Andrea Thomaz  (School of Interactive Computing, Georgia Institute of Technology)

Dr. Henrik Christensen (School of Interactive Computing, Georgia Institute of Technology)

Dr. Magnus Egerstedt (School of Electrical and Computer Engineering, Georgia Institute of Technology)

Dr. Michael Littman  (Department of Computer Science, Brown University)

Abstract

------------

Domestic robotics has grown dramatically in the past decade, with applications ranging from house cleaning to food service to health care.  A key challenge in deploying robots in these settings is the current dependence on a human expert to specify the appropriate goals, models, and algorithms for each target environment.  This dissertation meets this challenge by introducing a stochastic physics engine as the core model representation for a Reinforcement Learning Agent.  This approach allows us to leverage the appealing properties of Reinforcement Learning, in which an agent can be trained online using only rewards and sensor observations, for problems in mobile manipulation with real humanoid robots.

We define procedures for enabling a robot to autonomously decide when and how to update its beliefs about the stochastic engine parameters.  We then present an appropriate space of motor controllers for these models, and introduce ways to intelligently select when to use each controller based on the estimated model parameters.  Together these methods enable a robot to adapt to unfamiliar environments without human intervention.  We demonstrate our approach across a range of domestic tasks, starting with a simple table-top manipulation task, followed by a mobile manipulation task involving a single utility cart, and finally an open-ended navigation task with arbitrary obstacles impeding robot progress.