Title: Interactive Tracking and Action Retrieval to Support Human Behavior Analysis

Arridhana Ciptadi

Ph.D. Student

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: Tuesday, June 16th, 2015

Time: 2PM to 4PM EST

Location: Klaus 1212

Committee:

Dr. James M. Rehg, School of Interactive Computing, Georgia Tech (co-Advisor)

Dr. Gregory D. Abowd, School of Interactive Computing, Georgia Tech (co-Advisor)

Dr. Agata Rozga, School of Interactive Computing, Georgia Tech

Dr. Daniel Messinger, Department of Psychology, University of Miami

Dr. Pietro Perona, Division of Engineering and Applied Science,

California Institute of Technology

Abstract:

The goal of this thesis is to develop a set of tools for continuous

tracking of behavioral phenomena in videos to support human behavior

study. Current standard practices for extracting useful behavioral

information from a video are typically difficult to replicate and

require a lot of human time. For example, extensive training is

typically required for a human coder to reliably code a particular

behavior/interaction. Also, manual coding typically takes a lot more

time than the actual length of the video. The time intensive nature of

this process puts a strong limitation on the scalability of the study.

Furthermore, this makes it difficult for a third party to perform

replication study.

To address this issue, I have developed an efficient interactive

tracking and behavior retrieval system. These tools allow behavioral

researchers/clinicians to more easily extract relevant behavioral

information, and more objectively analyze behavioral data from videos. I

have demonstrated that my behavior retrieval system achieve

state-of-the-art performance in a preliminary experiment. In the

proposed work, I will perform a more thorough evaluation of this system

on a wider set of behaviors. I have also demonstrated that my

interactive tracking system is able to produce high-precision tracking

results with less human effort compared to the state-of-the-art. In the

proposed work, I will show how a proximity measure that is derived from

the results of interactive tracking can be used to predict attachment

classification in The Strange Situation, a protocol for studying infant

attachment security. In addition, I will also show how this measure

opens a new avenue of behavioral research by showing that it can be used

to quantify the consistency of infant behavior in the two reunion

episodes in Strange Situation.