Name: David Grimm

Master’s Thesis Defense Meeting
Date:  November 24, 2020
Time:  10:00AM
Location: https://us02web.zoom.us/j/85915191157
 
Advisor:
Jamie Gorman, Ph.D. (Georgia Tech)
 
Thesis Committee Members:

Jamie Gorman, Ph.D. (Georgia Tech)
Richard Catrambone, Ph.D. (Georgia Tech)

Rick Thomas, Ph.D. (Georgia Tech)

Nancy Cooke, Ph.D. (Arizona State)
 
Title: Dynamical Analysis and Modeling of Team Resilience in Human-Autonomy Teams

Abstract:

A resilient team would be proficient at overcoming sudden, unexpected changes by displaying a rapid, adaptive response to maintain effectiveness. To quantify resilience, I analyzed data from two different experiments examining performance of human-autonomy teams (HATs) operating in a remotely piloted aircraft system (RPAS). Across both experiments, the HATs experienced  a variety of automation and autonomy failure perturbations using a Wizard of Oz paradigm. Team performance was measured by the successful completion of simulated reconnaissance missions, a mission level team performance score, a coordination-based target processing efficiency (TPE) score to quantify team efficiency, and a ground truth resilience score (GTRS) to measure how teams performed during and following a failure. Different layers, composed of vehicle, operator controls, communication, and overall system layers, of sociotechnical elements of the system were measured across RPAS missions. To measure resilience, I used entropy and a root mean squared error (RMSE) metric across all system layers. I used these measures to examine the time taken to achieve extreme values of reorganization during a failure and the novelty of the reorganization, respectively, to quantify resilience. I hypothesized that faster times to achieve extreme values of reorganization during a failure would be correlated with all performance measures. Across both experiments, I found negative correlations of time taken to achieve extreme values of reorganization and novelty of reorganization with team performance measured using TPE, and positive correlations while using GTRS. Additionally, I found that teams displayed more reorganization in response to failures, but this was not pronounced for effective teams. In Experiment 2, I also found differential effects of training in the communication and control layers. I hope that these results can help inform the measurement and training of resilience in HATs through targeted team training, feedback, and real-time analysis applications.