Title:  Modeling Natural-Image Spaces for Single-Label Image Classification & Photo-Realistic Style Transfer and Directionally-Paired Principal Component Analysis for Estimation of Coupled Data

Committee:

Dr. Anthony Yezzi, ECE, Chair , Advisor

Dr. Mark Davenport, ECE

Dr. Patricio Vela, ECE

Dr. Frieder Ganz, Adobe

Dr. Shijie Deng, ISyE

Abstract: In this dissertation, we aim at a methodology that provides better explainability of machine-learning algorithms both under the linear-analysis framework and in the deep-learning domain. Under the linear-analysis framework, we extend the classic PCA for coupled yet partially observable test data. The proposed Directionally Paired Principal Component Analysis (DP-PCA) is the optimal linear model that performs dimension reduction and least-square regression between the coupled variable sets in the principal subspace, which leads to the lowest estimation errors at a faster speed and with less storage requirement compared to existing cross-decomposition methods. In the deep-learning domain, we provide a unified explanation on the behaviors of various machine-learning algorithms as well as the gap between human and machine perception with the proposed conceptual model of natural-image spaces. By formulating classification as a partition of the image space, we further provide a topological view of knowledge in machine perception by defining fundamental concepts including information, knowledge, beliefs and truths. We illustrate the benefits from the proposed conceptual image-space model and the topological view of knowledge in two concrete applications: single-label image classification and photo-realistic style transfer. In single-label image classification, we take advantage of the hidden information in the image spaces to enhance or preserve classification accuracy and verify geometric causes of adversarial examples following the properties of the image spaces. In photo-realistic style transfer, we re-partition the image space with artistic presets, which introduces style-awareness to classifiers and provides useful guidance signals that support global style transfer. By modeling the image space at the object level, we ensemble a content-aware local style transfer pipeline in which the proposed segmentation refinement module removes defects from inaccurate segmentation maps and supports feature blending at various levels.