Committee: 

Dr. Theodorou, Advisor

Dr. Vela, Co-Advisor     

Dr. Coogan, Chair

Dr. Rehg

Dr. Al-Regib

Abstract: The object of the proposed research is to develop safe and robust vision-based control algorithms for autonomous vehicles. In our robust vision-based imitation learning algorithms, we propose the use of Bayesian Neural Networks (BNNs), which provide both a mean value and an uncertainty estimate as output, to enhance the safety of learned control policies when a test-time input differs significantly from the training set. Furthermore, to quickly detect abnormal uncertain situations in vision-based control, we use Model Predictive Control (MPC) to learn how to focus on important areas of the visual input. This attention-based mechanism allows the system to more rapidly detect unsafe conditions when novel obstacles are present in the navigation environment. Another vision-based control algorithm, the PixelMPC with the Deep Optical Flow dynamics, robustifies the vision-based state estimation of the robot. This novel MPC algorithm allows us to predict both robot's optimal path and the path of a pixel-of-interest in the scene. By controlling a pixel with its learned optical flow dynamics, a robot can have better and stable visual information which results in a robust state estimation followed by robust path planning and control. The proposed algorithm is tested in a photorealistic simulation with a high-speed drone racing task.