Title:  Bridging Distributional Discrepancy with Temporal Dynamics for Video Understanding

Committee:

Dr. Ghassan AlRegib, ECE, Chair , Advisor

Dr. Zsolt Kira, CoC

Dr. Patricio Vela, ECE

Dr. Eva Dyer, BME

Dr. Yi-Chang Tsai, CEE

Abstract:

Video has become one of the major media in our society, bringing considerable interests in the development of video analysis techniques for various applications. Temporal Dynamic, which represents how information changes along time, is the key component for videos. However, it is still not clear how temporal dynamics benefit video tasks, especially for the cross-domain case, which is close to real-world scenarios. Therefore, the objective of this thesis is to effectively exploit temporal dynamics from videos to tackle distributional discrepancy problems for video understanding. To achieve this objective, firstly I proposed two approaches to exploit spatio-temporal dynamics: 1) Temporal Segment LSTM (TS-LSTM) and 2) Inceptionstyle Temporal-ConvNet (Temporal-Inception). Secondly, I collected two large-scale datasets for cross-domain action recognition: UCF-HMDBfull and Kinetics-Gameplay to facilitate cross-domain video research, and proposed Temporal Attentive Adversarial Adaptation Network (TA3N) to simultaneously attend, align and learn temporal dynamics across domains. Finally, to utilize temporal dynamics from unlabeled videos for action segmentation, I proposed Self-Supervised Temporal Domain Adaptation (SSTDA) to jointly align cross-domain feature spaces embedded with local and global temporal dynamics.