Mackenzie Lau
(Advisor: Prof. Mavris]

will defend a doctoral thesis entitled,

A Methodology to Enhance Quantitative Technology Evaluation Through Exploration of Employment Concepts for Systems-of-Systems

On

Tuesday, July 6 at 8:00 a.m. EDT
https://bluejeans.com/521165895/7233

Abstract
The process of designing a new system has often been characterized as purely technological in nature, where the infusion or synthesis of new technologies forms the basis of advancement. However, recent trends in design and analysis methodologies have begun to shift away from the narrow scope offered by materiel-focused approaches. One trend has been to increase the analysis scope from the system level to that of the system-of-systems, allowing for the exploration of non-materiel solutions to capability gaps which would otherwise be resolved through technological means. This class of solutions can reduce the cost of the effort to close a capability gap by mitigating the need to procure new systems to achieve desired levels of performance. Employment concepts, or the ways in which technological means are utilized to achieve an end, are a type of non-materiel solution which can enhance existing, evolutionary, and revolutionary systems.

In the past, the task of experimenting with employment concepts has largely been left to operators after systems have been designed and fielded. This necessarily calls into question whether the chosen design adequately accounted for the possibility of innovation employment concepts which operators might discover, and a review of historical and modern accounts indicated a persistent shortcoming in this area. Attempts can be made to bring the empirical knowledge possessed by skilled operators upstream in the design process. However, care must be taken to ensure such attempts do not introduce unwanted bias into the design process. Furthermore, the capacity for human operators to capitalize on the potential benefits of a given technology may be limited when dealing with revolutionary concepts to which prior knowledge is not applicable. Each of these complicating factors is exacerbated by the system-of-systems, where changes in the interactions between entities can significantly influence outcomes. Problems of this type necessitate exploration and analysis of employment concepts for several entities simultaneously, not only that or those on which the design effort is focused.

This research sought to address the challenges in exploring employment concepts in the system design process within the context of a system-of-systems. A characterization of the problem identified several gaps in existing techniques and methodologies, particularly with respect to the representation, generation, and evaluation of alternative employment concepts. Relevant theories, including behavioral psychology, control theory, and game theory, were identified to facilitate closure of these gaps. However, these theories also introduced technical challenges which had to be overcome. These challenges stemmed from systematic problems such as the curse of dimensionality, temporal credit assignment, and the complexities of system interactions. A candidate approach was identified through thorough review of available literature: Multi-agent reinforcement learning with augmented state spaces. Experiments show the proposed approach could be used to generate effective models of behavior which perform better than existing models from literature on a canonical problem. It was further shown that models produced by this method could achieve robust performance in scenarios where multiple interacting agents were learning simultaneously and in direct competition with one another. Lastly, it was shown how incorporation of design variables into the state space could allow models to learn policies which were effective across a continuous, multi-dimensional design space. All of these results were obtained without reliance on prior knowledge, mitigating the potential for unwanted bias in the analysis process and supporting the applicability of the methodology to problems where convenient baselines are not available. Lastly, the methodology was applied to the design of a fighter aircraft for one-on-one, gun-only air combat engagements. Analysis revealed distinct trends in the design space and sensitivity to the choice of behavior model used. The trends generally agreed with common knowledge about the problem, and provided additional insights into how small changes in the decisions made by each agent could influence the outcome of the engagement. This demonstrated how the new methodology could be used to support analyses of complex and practical problems, and enable exploration of the coupled spaces of tactics and technologies which would otherwise be infeasible.

Committee

• Prof. Dimitri Mavris – School of Aerospace Engineering, Georgia Institute of Technology (advisor)
•  
• Dr. Michael Steffens– School of Aerospace Engineering, Georgia Institute of Technology
•  
• Mr. James Zeh – Air Force Research Laboratory
•  
• Prof. Daniel Schrage – School of Aerospace Engineering, Georgia Institute of Technology
•  
• Dr. Kelly Griendling – School of Aerospace Engineering, Georgia Institute of Technology
•