Ph.D. Thesis Defense by

Ioannis Exarchos

(Advisor: Prof. Panagiotis Tsiotras)

“Stochastic Optimal Control – An Fbsde Sampling Approach”

Wednesday, November 08, 2017 @ 10 a.m.
Montgomery Knight Building, Room 317

Abstract:
Stochastic optimal control lies within the foundation of mathematical control theory ever since its inception. Its usefulness has been proven in a plethora of engineering applications, such as autonomous systems, robotics, neuroscience, and financial engineering, among others. Specifically, in robotics and autonomous systems, stochastic control has become one of the most successful approaches for planning and learning, as demonstrated by its effectiveness in many applications, such as control of ground and aerial vehicles, articulated mechanisms and manipulators, and humanoid robots.  In computational neuroscience and human motor control, stochastic optimal control theory is the primary framework used in the process of modeling the underlying computational principles of the neural control of movement. Furthermore, in financial engineering, stochastic optimal control provides the main computational and analytical framework, with widespread application in portfolio management and stock market trading.

By and large, prior work on stochastic control theory and algorithms imposes restrictive conditions such as differentiability of the dynamics and cost functions, and furthermore requires certain assumptions involving the control authority and stochasticity to be met. Thus, it may only address special classes of systems. The goal of this research is to establish a framework that goes beyond these limitations. In particular, we propose a learning stochastic control framework which capitalizes on the innate relationship between certain nonlinear PDEs and Forward and Backward SDEs (FBSDEs) demonstrated by a nonlinear version of the Feynman-Kac lemma. By means of this lemma, we are able to obtain a probabilistic representation of the solution to the nonlinear Hamilton-Jacobi-Bellman equation, expressed in form of a system of decoupled FBSDEs. This system of FBSDEs can then be simulated by employing linear regression techniques. The overall approach will allow us to learn the value function in stochastic optimal control problems with highly nonlinear dynamics.  In addition, the proposed approach exhibits the following characteristics:

• It performs stochastic control and trajectory optimization without linearization of the dynamics and quadratic approximations of the cost functions.

• Yields nonlinear feedback control policies that offer higher performance than their traditional trajectory optimization counterparts.

• Is based on sampling, scalable, and therefore directly applicable to high dimensional systems.

• Expands the class of systems currently addressed by traditional stochastic optimal control methods.

The framework we develop within this thesis addresses several classes of stochastic optimal control, such as L2 , L1 , risk sensitive control, as well as differential games. Both fixed final time and first exit settings are considered.

Committee Members:
Prof. Panagiotis Tsiotras (Advisor)
Prof. Evangelos Theodorou (AE)
Prof. Wassim Haddad (AE)
Prof. Hao-min Zhou (MATH)
Prof. Ionel Popescu (MATH)