PhD Defense by Kaushik Subramanian

Event Details
  • Date/Time:
    • Wednesday May 17, 2017
      10:00 am - 12:00 pm
  • Location: CCB 345
  • Phone:
  • URL:
  • Email:
  • Fee(s):
  • Extras:
No contact information submitted.

Summary Sentence: Policy-based Exploration for Efficient Reinforcement Learning

Full Summary: No summary paragraph submitted.

Title: Policy-based Exploration for Efficient Reinforcement Learning


Kaushik Subramanian

Ph.D. Candidate

School of Interactive Computing

College of Computing

Georgia Institute of Technology


Date: Wednesday, May 17th 2017

Time: 10 AM to 12 NOON ET

Location: CCB 345




Dr. Charles Isbell (Advisor, School of Interactive Computing, Georgia Institute of Technology)

Dr. Andrea Thomaz (Co-Advisor, Department of Electrical and Computer Engineering, University of Texas at Austin)

Dr. Mark Riedl (School of Interactive Computing, Georgia Institute of Technology)

Dr. Thad Starner (School of Interactive Computing, Georgia Institute of Technology)

Dr. Peter Stone (Department of Computer Science, University of Texas at Austin)




Reinforcement Learning (RL) is the field of research focused on solving sequential decision-making tasks modeled as Markov Decision Processes. Researchers have shown RL to be successful at solving a variety of problems like system operations (logistics), robot tasks (soccer, helicopter control) and computer games (backgammon); however, in general, it is well-known that standard RL approaches do not scale well with the size of the problem. The reason this problem arises is that RL approaches rely on obtaining samples useful for learning the underlying structure. In this work we tackle the problem of smart exploration in RL, by using human interaction. We propose policy-based methods that serve to effectively bias exploration towards important aspects of the domain.


Reinforcement Learning agents use function approximation methods to generalize over large and complex domains. One of the most well-studied approaches is using linear regression algorithms to model the value function of the decision-making problem. We introduce a policy-based method that uses statistical criteria derived from linear regression analysis to bias the agent to explore samples useful for learning. We show how we can learn exploration policies autonomously and from human demonstrations (using concepts of active learning) to facilitate fast convergence to the optimal policy. We then tackle the problem of human-guided exploration in RL. We present a probabilistic method which combines human evaluations, instantiated as policy signals, with Bayesian RL. We show how this approach provides performance speedups while being robust to noisy, suboptimal human signals. We also present an approach that makes use of some of the inherent structure in the exploratory human demonstrations to assist Monte Carlo RL to overcome its limitations and efficiently solve large-scale problems. We implement our methods on popular arcade games and highlight the improvements achieved using our approach. We show how the work on using humans to help agents efficiently explore sequential decision-making tasks is an important and necessary step in applying Reinforcement Learning to complex problems

Additional Information

In Campus Calendar

Graduate Studies

Invited Audience
Faculty/Staff, Public, Undergraduate students
Phd Defense
  • Created By: Tatianna Richardson
  • Workflow Status: Published
  • Created On: May 15, 2017 - 9:22am
  • Last Updated: May 15, 2017 - 9:22am