The University of Minnesota’s Hyun Soo Park presents “Behind the Scenes: Decoding Intent from First Person Video” as part of the IRIM Robotics Seminar Series. The event will be held in the Marcus Nanotechnology Building, Rooms 1116-1118, from 12-1 p.m. and is open to the public.

Abstract

A first-person video records not only what is out in the environment but also what is in our head (intention and attention) at the time via social and physical interactions. It is invisible but it can be revealed by fixation, camera motion, and visual semantics. In this talk, I will present a computational model to decode our intention and attention from first-person cameras when interacting with (1) scene and (2) people.

A person exerts his/her intention through applying physical force and torque to scenes and objects, which effects in visual sensation. We leverage the first person visual sensation to precisely compute force and torque that the first person experienced by integrating visual semantics, 3D reconstruction, and inverse optimal control. Such visual sensation also allows associating with our past experiences that eventually provide a strong cue to predict future activities. When interacting with other people, social attention is a medium that controls group behaviors, e.g., how they form a group and move. We learn the geometric and visual relationship between group behaviors and social attention measured from first-person cameras. Based on the learned relationship, we derive a predictive model to localize social attention from a third-person view. 

Bio

Hyun Soo Park is an assistant professor in the Department of Computer Science and Engineering at the University of Minnesota. He is interested in understanding human visual sensorimotor behaviors from first-person cameras. Prior to joining the UMN, he was a postdoctoral fellow in the GRASP Lab at the University of Pennsylvania. Park earned his Ph.D. from Carnegie Mellon University.