Matthew Guckenberger
(Advisor: Prof. Mavris)

will defend a doctoral thesis entitled,

A Methodology for Reducing Uncertainty in Performance Evaluations for Increasingly Automated Systems

On

Wednesday, July 7 at 8:00 a.m.
https://bluejeans.com/655040323/7553

Abstract
As systems move through the levels of automation on the way to autonomy, there is a growing and unsustainable amount of uncertainty within the early development process technology evaluations. The role of the operator is changing from sole actor to a shared supervisor/actor role. This change requires new modeling techniques in operational studies to provide the analytics for assessing system performance. Studies on unmanned ground vehicle operators and recent crashes being partially blamed on automation technologies demonstrate the need to measure and assess operator awareness and workload. Overcoming these challenges requires an assessment early in the design cycle for operator awareness and workload. This modeling methodology integrates concepts from cognitive engineering into operations analysis to better capture and analyze the effectiveness of increasingly automated systems. An agent-based model is created using Operational Event Sequence Diagrams and concepts from situation awareness research to guide agent formulation. The agent rule set is then mapped to the NASA Task Load Index scales to provide a dynamic output throughout the simulation. The methodology is demonstrated using two case studies, an assessment of driving technologies and an intelligence, surveillance, and reconnaissance (ISR) mission utilizing an unmanned aerial combat vehicle (UCAV).  The driving case study demonstrates the steps of the methodology and is used for benchmarking.  The virtual results showed the same trends and similar normalized differences to experimental data found in literature. The ISR mission demonstrates the utility of the methodology in the aerospace domain, specifically assessing manned-unmanned teaming. The dynamic workload measurement is the first step in a framework that will enable automation technologies to be traded during the conceptual design phase. The diligent mapping of actions between the automation and operator, along with these new awareness and workload metrics, is required if operation’s models are expected to provide decision makers with the analytics necessary for assessing heavily automated systems.

Committee

• Prof. Dimitri Mavris – School of Aerospace Engineering (advisor)
• Prof. Daniel Schrage – School of Aerospace Engineering
• Dr. Alicia Sudol – School of Aerospace Engineering
• Prof. Mariel Borowitz – School of International Affairs
• Dr. Charles Domercant – Senior Research Engineer, Georgia Tech Research Institute