Strategies for Handling Large Vocabulary and Data Sparsity Problems for Tamil Speech Recognition

Abstract

This thesis focuses on the design and development of every building block of a very large vocabulary, continuous speech recognition (LVCSR) system and various experiments conducted in order to enhance its performance. To our knowledge, this is the first report of development of such a full-fledged Tamil LVCSR system, since we could not find any journal or even refereed conference papers building a Tamil LVCSR system and reporting recognition results on large scale open-source speech recognition test datasets. A large read speech corpus of 217 hours has been collected and annotated at the sentence level for the development of the LVCSR system. Out of this, 160 hours of data has been used for training the LVCSR, 50 hours as the test set, and the publicly available 7 hours of OpenSLR-65 data released by Google is used as the development set. The major contributions of the thesis are: Collection of a large amount of Tamil speech and text data, editing the transcriptions to match the spoken utterances and using them to develop a deep neural network (DNN) and graphical model-based Tamil LVCSR system. Handling the unlimited vocabulary problem in Tamil by proposing subword modeling technique using novel subword dictionary creation and word segmentation techniques implemented efficiently using weighted finite state transducer (WFST) framework. Addressing the data sparsity problem by leveraging data from multiple low and medium resourced languages by pooling data using novel phone/senone mapping techniques and training a multitask DNN (MT-DNN). Proposing a novel coactivation loss for speaker-adapting the DNN using asymptotic Bayesian approximation through Laplace approximation, by using mean and covariance statistics of the activation values at all the hidden layers of the speaker-independent DNN. Studying the use of scattering transform features in acoustic modeling and proposing a DNN architecture inspired by it to jointly perform feature extraction and acoustic modeling from raw speech signal...