Hyeokhyen Kwon
Ph.D. student in Computer Science
School of Interactive Computing
College of Computing
Georgia Institute of Technology
Date: Monday, October 19th, 2020
Time: 10:00 AM to 12:00 PM (EST)
Location: https://bluejeans.com/3537894193
Committee:
Dr. Gregory D. Abowd (Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Thomas Ploetz (Co-Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Thad Starner – School of Interactive Computing, Georgia Institute of Technology
Dr. Irfan Essa – School of Interactive Computing, Georgia Institute of Technology
Dr. Nicholas D. Lane – Dept. of Computer Science & Tech., University of Cambridge
Abstract:
Wearable Inertial Measurement Unit (IMU)-based human activity recognition is at the core of continuous monitoring for human well-being, which can detect precursors of health risks in everyday life. Conventionally, wearable sensor data is collected from recruited users, where user engagement is expensive, and the annotation is time-consuming. Due to the lack of large-scale labeled datasets, the wearable-based human activity recognition model has yet to experience significant improvements in recognition performance. To tackle the scale limitations in the wearable sensor dataset, this dissertation proposes a novel approach, which aims at harvesting existing video data from virtually unlimitedly large repositories, such as YouTube. I introduce an automated processing pipeline that integrates existing computer vision and signal processing techniques to convert human activity videos into virtual IMU data streams. I show how the virtually-generated IMU data improves the performance of various models on known human activity recognition datasets. I also proposed approaches to improve the quality of the generated virtual IMU data and decrease the domain gap between virtual and real IMU data. To further improve the recognition accuracy, I discuss a novel model training approach to handle human activity annotation noise in video datasets. This dissertation shows the promise of using video as a novel source for human activity recognition with wearables, representing a paradigm shift for deriving a robust human activity recognition system.
]]>