Title:  Modeling natural-image spaces for enhanced robust image classification and non-destructive photo style transfer

Committee: 

Dr. Yezzi, Advisor

Dr. Davenport, Chair

Dr. Vela

Abstract:

The objective of the proposed research is to study and model the space of natural images at a micro-level so that we can better guide machine-learning algorithms with human-level abstraction in solving computer-vision tasks including image classification and non-destructive style transfer for photographs. The motivation of this research is that the prevailing "image-manifold” model cannot sufficiently explain the uncertainties in machine learning, and a unified understanding of the behaviors of machine-learning algorithms is missing. The research aims to explain and mitigate the uncertainty issues such as dataset bias and adversarial examples with thoughtful strategies that account for the geometry and topology of the natural-image spaces. For image classification, the first strategy will handle unexpected classes of inputs by augmentation using “off-the-manifold” samples such as random-noise. The second strategy will defend adversarial examples with smoothing. The third strategy will introduce adaptive equivalence classes that are connected via a table/tree structure to tackle discontinuity in classifiers to address the fact that neural networks primarily approximate continuous mappings and tend to divide an input space into path-connected regions. For non-destructive photo-style transfer, we will design artistic features with human-level abstraction that can group scattered photos in similar styles  with fewer uncertain regions. To further assist the transfer, we will build a navigation system based on the logits from multiple discriminators, each corresponding to a direction from a source to the artist distribution.