Title:  Reliable Sensor Intelligence in Unreliable Environment

Committee: 

Dr. Mukhopadhyay, Advisor    

Dr. Krishna, Chair

Dr. Kim

Abstract: The objective of this research is to design a sensor intelligence that is reliable in a resource constraint, unreliable environment. The research particularly focuses on improving the reliability of vision based intelligent sensor system using digital pixels to perform complex vision tasks. The research suggests three approaches to achieve this goal; (1) enhance the robustness of the task, (2) adopting an early warning to predict task failure, and (3) employing uncertainty to estimate task reliability. First, to improve the robustness of the task, the research studies a DNN based fully spatiotemporal preprocessor that removes the spatiotemporal corruptions of input data and enhances the spatiotemporal task performance. Moreover, the research presents a digital pixel-DNN cross-layer simulation methodology to design a robust DNN to sensor hardware derived noise. Second, the concept of early warning generation is introduced to predict unreliable task output in advance and control the sensor operation to avoid system failure. Third, lightweight uncertainty estimator is suggested to adopt the DNN model uncertainty as a measure of task reliability without prohibitive computation from stochastic DNN. To complete the research, the remainder of the study is to develop a lightweight uncertainty estimator for more complex tasks and an uncertainty guided adaptive sensor system.