Title:  Constrained PDE Optimization Methods for Motion Segmentation and Layer Extraction

Committee:

Dr. Anthony Yezzi, ECE, Chair , Advisor

Dr. Ghassan AlRegib, ECE

Dr. Patricio Vela, ECE

Dr. Erik Verriest, ECE

Dr. Sung Ha Kang, Math

Abstract:

A layered representation of images allows us to capture motion, shape, appearance and occlusion structure without going into the complexity of a full 3D representation of the scene. A unified computational framework which integrates much of the current and prior work on layered models, would aid our understanding and development of layer extraction algorithms. The objective of the proposed research is to present a unified variational framework for building generative layered models that have flexibility in modeling shape, appearance, motion and occlusion structure for objects in a set of images. This problem is approached in the framework of PDEs and calculus of variations. More
specifically the strategy is developed using active contours due to their ability to capture shape and topology in an arbitrarily flexible way. The proposed approach builds on the classical Mumford Shah style segmentation by adapting it to to a layered framework instead. A novelty of this modeling technique is that it relaxes the brightness constancy constraint for moving objects making the model a better fit for tracking objects in most real life scenarios. The technique links to the simplification of an earlier approach to layering (Jackson[2008] ) which was not further developed since its proposal in 2008.  Their usage of diffeomorphic maps is simplified to active contours which significantly reduces the computational complexity of the model yet still provides enough modeling richness for pertinent applications. More importantly a subtle yet fundamental modeling flaw in this earlier work is discovered which accidentally penalized layer occlusion as images with a more heterogeneous appearance were used (which is where the model should have excelled). The cause of this is traced to a bias in the original formulation that unintentionally penalized layer occlusion thereby producing the effect. The problem is resolved by replacing the prior joint shape and appearance optimization strategy with an alternative Lagrangian style constrained shape optimization subject to PDE based appearance constraints. Bringing out the true potential of the proposed technique, MOVE (Moving Object Video Encoding), a novel technique for representing video is introduced.