Title:  Exploiting Spatio-Temporal Dynamics for Cross-Domain Video Understanding

Committee: 

Dr. AlRegib, Advisor    

Dr. Vela, Chair

Dr. Kira

Abstract:

The objective of the proposed research is to effectively extract spatio-temporal information from videos for cross-domain video tasks. Video has become one of the major media in our society, bringing considerable interests in the development of video analysis techniques for various applications. Temporal information, which represents how information changes along time, is the key component for videos. However, it is still not clear how temporal information benefits video tasks, especially for the cross-domain case, which is close to real-world scenarios. To achieve this objective, firstly I addressed this question for video classification: given the spatial and motion feature representations over time, what is the best way to exploit the temporal information? Two different approaches were investigated to exploit spatio-temporal dynamics: 1) Temporal Segment LSTM (TS-LSTM) and 2) Inception-style Temporal-ConvNet (Temporal-Inception). Both approaches were shown achieving comparable state-of-the-art performances, without requiring extensive temporal augmentation. The goal of my thesis is to exploit spatio-temporal information to investigate the under-explored domain shift problem in videos. Since most previous work only evaluates the performance on small-scale datasets with little domain discrepancy, I collected two large-scale datasets for video domain adaptation: UCF-HMDB_full and Kinetics-Gameplay to facilitate cross-domain video research, and investigated various domain adaptation (DA) integration approaches for videos. In the proposed work, I will study how to simultaneously align and learn spatio-temporal dynamics for various video DA tasks.