678283 event 1731054760 1731054778 <![CDATA[Ph.D. Proposal Oral Exam - Pin-Jui Ku]]> Title:  Incorporating Consistency Constraint with Deep Models for Robust Phase Reconstruction and Speech Enhancement

Committee: 

Dr. Lee, Advisor   

Dr. Heck, Chair

Dr. Anderson

]]> In this proposal, our main goal is to develop a novel phase estimation framework to address the challenges of Phase Reconstruction (PR) and validate its effectiveness across various state-of-the-art (SOTA) Speech Enhancement (SE) models. Our preliminary research shows that the quality of the speech signal can be preserved as long as the magnitude and phase spectrogram are consistent. Building on these findings, we propose a novel phase loss function based on the consistency constraint, which achieves superior performances to existing phase loss functions in the SE task. In our proposed research, we will begin by exploring the integration of the proposed consistency constraint with the geometric constraint, another key relationship between magnitude and phase spectra in speech enhancement, as a potential way to further boost phase reconstruction performances. We will then apply our framework to various modern SE models with SOTA performances, including those utilizing deep state-space architectures, to examine how it interacts with different models. Lastly, we aim to extend the framework to large-scale generative model pretraining for further improvements with a thorough theoretical and empirical validation across a range of speech restoration tasks.

]]> <![CDATA[]]> 434371 1788 102851 1808