Date: 8 May 2012 (revised)
Time: 1:00 - 3:00 pm (revised)
Location: Klaus 1116W (revised)
Committee:
Abstract:
My Thesis: Integrating modular reinforcement learning (MRL) into a programming language supports adaptive agent software engineering. There are three claims implied in this thesis statement: (1) there is a such thing as MRL in a software engineering sense, (2) integrating MRL into a programming language is feasible, and (3) integrating MRL into a programming language is useful to software engineers writing adaptive software agents.
Modular reinforcement learning decomposes a reinforcement learning agent into components that solve subproblems of the total problem faced by an agent. Hierarchical reinforcement learning (HRL), which decomposes problems temporally into subtasks, is well developed. MRL, which decomposes problems into concurrent subproblems, is still nascent. Existing approaches to MRL are not modular in a software engineering sense because inter-component reward coupling prevents reuse. This dissertation will demonstrate the reward coupling problem and contribute a solution in the form of a reformulation of MRL and an algorithm that implements it.
Our goal is to support practical software engineering. The best way to support software engineering is with practical, usable programming languages. This dissertation will contribute a programming language, implemented as a Scala library and asosciated idioms and design patterns, called AFABL -- A {Friendly|Flexible} Adaptive Behavior Language -- that integrates MRL, making MRL useful to software engineers writing practical adaptive agent software.
Finally, we will apply AFABL to non-player character (NPC) programming in games and agent simulations to demonstrate its usefulness to software engineers writing adaptive software agents. This application of AFABL to practical software engineering problems will distinguish AFABL from previous work in integrating RL into programming languages such as ALisp.