Title:  Efficient and Reliable Edge Vision with In-Memory and Near-Pixel Compute

Committee: 

Dr. Yu, Advisor   

Dr. Shaolan Li, Chair

Dr. Hao

Abstract: The objective of the proposed research is to accelerate heavy vision workloads in edge devices by designing a near-pixel compute-in-memory (CIM) system. While CIM offers highly parallel processing with reduced data movements, the network accuracy loss due to analog compute remains as a challenge. To address the reliability issues, two generations of CIM inference macros based on resistive random access memory (RRAM) have been taped-out and validated to demonstrate multiple techniques that improve the reliability of such systems, while maintaining competitive energy efficiency and throughput. These techniques include on-chip write-verify, offset-cancelling ADC references, in-situ error correction for CIM, and embedded security for on-chip synaptic weights. In addition, a near-pixel temporal filtering network for autonomous driving is implemented to prevent redundant frames from being transmitted to downstream processors, thereby reducing the data transmission cost and improving bandwidth. The proposal presents a research plan to combine in-memory and near-pixel compute into a full system. The proposed research captures the benefits of these two emerging compute paradigms to allow massively parallel processing at the pixel frontend with high reliability and low power budget, while avoiding transmission of full-resolution image frames to backend processors.