Ph.D. Thesis Proposal by

(Advisor: Prof. Panagiotis Tsiotras)
2 p.m., Wednesday, March 29
Weber Building, Room 200

ABSTRACT:
Stochastic optimal control lies within the foundation of mathematical control theory ever since its inception. Its usefulness has been proven in a plethora of engineering applications, such as autonomous systems, robotics, neuroscience, and financial engineering, among others. Specifically, in robotics and autonomous systems, stochastic control has become one of the most successful approaches for planning and learning, as demonstrated by its effectiveness in many applications, such as control of ground and aerial vehicles, articulated mechanisms and manipulators, and humanoid robots. In computational neuroscience and human motor control, stochastic optimal control theory is the primary framework used in the process of modeling the underlying computational principles of the neural control of movement. Furthermore, in financial engineering, stochastic optimal control provides the main computational and analytical framework, with widespread application in portfolio management and stock market trading. By and large, prior work on stochastic control theory and algorithms imposes restrictive conditions such as differentiability of the dynamics and cost functions, and furthermore requires certain assumptions involving the control authority and stochasticity to be met. Thus, it may only address special classes of systems. The goal of this research is to establish a framework that goes beyond these limitations. In particular, we propose a learning stochastic control framework which capitalizes on the innate relationship between certain nonlinear PDEs and Forward and Backward SDEs (FBSDEs) demonstrated by a nonlinear version of the Feynman-Kac lemma. By means of this lemma, we are able to obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman equation, expressed in form of a system of decoupled FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. The overall approach will allow us to learn the value function in stochastic optimal control problems with highly nonlinear dynamics.

In addition, the proposed approach should exhibit the following characteristics:

• Perform stochastic control and trajectory optimization without linearization of the dynamics and quadratic approximations of the cost functions.
• Find nonlinear feedback control policies that yield higher performance than their traditional trajectory optimization counterparts.
• Be based on sampling, scalable, and therefore directly applicable to high dimensional systems, and able to accommodate parallel computation.
• Expand the class of systems currently addressed by traditional stochastic optimal control methods.

The framework we propose to develop within this thesis will address several classes of stochastic optimal control, such as L2 , L1 , game theoretic and risk sensitive control.

Committee Members:

• ·         Prof. Panagiotis Tsiotras (Advisor)
• ·         Prof. Evangelos Theodorou (AE)
• ·         Prof. Hao-min Zhou (MATH)