"Cortically-Coupled Computing for Image Search"

Paul Sajda, PhD

Columbia University

Our visual systems are amazingly complex information processing machines. Using our brain's visual system we can recognize objects at a glance, under varying pose, illumination, and scale, and are able to rapidly learn and recognize new configurations of objects and exploit relevant context even in highly cluttered scenes. However our brains are subject to fatigue and have difficulty finding patterns in high-dimensional feature spaces that are often useful representations for multimedia data. In this talk I will describe our work in developing a synergistic integration of human visual processing and computer vision via a novel brain computer interface (BCI). Our approach, which we term cortically-coupled computer vision (C3Vision), uses non-invasively measured neural signatures from the electroencephalogram (EEG) that are indicative of user intent, interest and high-level, subjective and rapid reactions to visual and multimedia data. I will describe several system designs for C3Vision and current applications that are being developed for government and commercial applications.