Title:  Improving Mispronunciation Detection And Enriching Diagnostic Feedback For Non-Native Learners Of Mandarin

Committee: 

Dr. Lee, Advisor  

Dr. Anderson, Chair

Dr. Moore

Abstract:

The objective of the proposed research is to improve mispronunciation detection of Mandarin and enrich diagnostic feedback for second language learners. The problem is tackled from the perspective of acoustic modeling and verification of phones and tones. For the acoustic modeling part, speech attributes and soft targets are respectively proposed to help resolve phone and tone's hard-assignments labels, which are not optimal for describing irregular non-native pronunciations. Subsequently, multi-source information or better trained acoustic model can provide more accurate features for mispronunciation detectors. For the verification part, pronunciation representation, usually calculated by frame-level averaging in a DNN, is now learned by BLSTM, which directly uses sequential context information to embed a sequence of pronunciation scores into a pronunciation vector to improve the performance of mispronunciation detectors. Finally, with the help of posterior scores generated by different classifiers, we can visualize non-native mispronunciations and provide comprehensive feedback, including articulation manner, place, and pitch contour-related diagnostic information, to help L2 learners.