Title: Scalable and Safe Deep Learning Architectures and Algorithms for Stochastic Optimal Control using Forward-Backward Stochastic Differential Equations 
 

Date: Monday, November 1, 2021

Time: 10:00 am – 12:00 pm EST

Location (Virtual): Teams Meeting Link

Location (In person): Montgomery Knight Building, MK 317 (conference room)

Marcus A. Pereira 

Robotics Ph.D. Student 

Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Georgia Institute of Technology

Committee:

Dr. Evangelos A. Theodorou (Advisor) – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Enlu Zhou – H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Samuel Coogan – School of Electrical and Computer Engineering, Georgia Institute of Technology

Dr. Yongxin Chen – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Kyriakos G. Vamvoudakis – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Ioannis Exarchos - Microsoft 

Abstract:

Stochastic Optimal Control in continuous-time requires solving the Hamilton-Jacobi-Bellman equation which suffers from the well-known curse-of-dimensionality. Instead of directly attempting to solve the HJB, one can obtain probabilistic representations of the solution via the Nonlinear Feynman-Kac lemma that relates the unique solution of the HJB to that of a system of Foward-Backward Stochastic Differential Equations or FBSDEs. This work develops novel algorithms that leverage the function approximation capabilities of deep neural networks to solve FBSDEs and empirically demonstrates that the framework is immune to compounding errors in comparison to past FBSDE works and is therefore, scalable. The resulting deep FBSDE framework is memory efficient, provides temporally smoother controls and can be employed for long time-horizons owing to the underlying Long-Short Term Memory network architecture. Starting from a basic Stochastic Optimal Control problem, the framework is extended to problem formulations such as systems with control-multiplicative noise, systems with dynamics non-affine with respect to the control and for non-quadratic control cost functions, for safety critical tasks which employs stochastic control barrier functions and for L1 stochastic optimal control in minimum-fuel aerospace applications. Each problem formulation is accompanied with necessary structural modifications to the deep learning architecture to enable end-to-end learning. Two directions of future research are then proposed which, (i.) improve scalability and (ii.) improve applicability of the framework. The first employs the Alternating Direction Method of Multipliers to offer a scalable solution for large-scale safe multi-agent stochastic optimal control and the second direction aims to extend the framework to systems with unknown dynamics using recent techniques borrowed from model-based reinforcement learning.