Title:  Discriminative and Adaptive Training od Deep Models for Robust Speech Recognition and Understanding

Committee: 

Dr. Juang, Advisor       

Dr. Lee, Chair

Dr. Moore

Abstract:

The objective of the proposed research is to build a robust speech recognition and understanding system through discriminative and adaptive training of the deep acoustic models. The goal is achieved through the following approaches: 1. To achieve accurate keyword spotting on conversational speech, the non-uniform error cost minimum classification error objective is used to discriminatively train the bi-direction long short-term memory (BLSTM)-recurrent neural network (RNN) acoustic model so that the errors of only keywords are minimized.  2. To generate semantically accurate word lattices for topic spotting, minimum semantic error cost objective function is proposed to train the BLSTM-RNN acoustic model, in which the expected semantic error cost of all possible word sequences on the lattices is minimized given the reference.  3. To cope with the mismatched training and test conditions in automatic speech recognition (ASR), domain separation networks are used for the unsupervised adaptation of the deep neural network acoustic models through adversarial multi-task training.  4. To achieve robust far-field ASR, beamforming is performed over speech signal acquired from multiple microphones. An LSTM-RNN is used to adaptively estimate the real-time beamforming filter coefficients to cope with non-stationary environmental noise and dynamic nature of source and microphones positions. The adaptive LSTM beamformer is jointly trained with a deep LSTM acoustic model to predict senone (tri-phone state) labels.