Title:  Robust Deep Vision-based Planning and Control Algorithms with Probabilistic Learning

Committee:

Dr. Evangelos Theodorou, AE, Chair, Advisor

Dr. Patricio Vela, ECE, Co-Advisor

Dr. Ghassan AlRegib, ECE

Dr. Samuel Coogan, ECE

Dr. James Rehg, CoC

Dr. Kyriakos Vamvoudakis, AE

Abstract: Decision making for safety-critical systems is challenging due to performance requirements with significant consequences in the event of failure. Data-driven planning and control methods, e.g. using deep neural networks, are generally not used for safety-critical systems as they can behave in unexpected ways in response to novel or corrupted inputs. This thesis studies how to safely deploy deep learning-based path planning and control algorithms to safety-critical systems (e.g. autonomous cars and drones). This thesis investigates the robot learning from demonstration paradigm, mainly imitation learning and inverse reinforcement learning. A novel system identification approach is also proposed to learn the dynamics of the optical flow. By combining computer vision, probabilistic deep learning, and model predictive control, we can 1) detect uncertain situations and stop the system before it fails, 2) quantify the uncertainty of the deep neural network model's prediction to plan a safer path for the system, and 3) optimize a trajectory to achieve more visual information and more robust state estimation. The proposed algorithms are tested in challenging environments including offroad racing tracks and a simulated dense traffic highway for autonomous driving and a photorealistic sensor simulated environment for drone racing.