Analysis and Implementation of Hybrid Learning Models for Vocal Emotion Conversion

Abstract

This thesis focuses on investigating, analyzing, designing, and implementing effective feature mapping strategies for neutral to emotional voice conversion. Majority of existing methods in literature synthesize one or more aspects of emotion from singlespeaker/ single-language training sets, but scalability is a significant issue in multispeaker emotion conversion. The widely varying nuances of expression from multiple newlinespeakers and cross-cultural backgrounds need to be incorporated for building speaker independent models. Non-availability of a large amount of emotional training data with parallel text also challenges the expansion of emotion conversion systems to multiple languages. The thesis aims to solve the above problems in generating emotional speech by employing intelligent soft computing techniques and signal processing aspects. The investigation of various methods for mapping emotionally relevant features is performed and validated in multiple languages. Here the spectral features we used are mel cepstral coefficients with different training methodologies for parallel and non-parallel data scenarios. The various modules shaping the characteristics of speech emotion and their useful integrations are focused, which led to the following contributions. The most significant feature characterizing emotion in any language is the prosody (fundamental frequency (F0), duration and intensity) or speech rhythm. As the first step in emotion conversion system design, a feasibility study for the prosodic modification in five selected languages for a multilingual scenario are explored, designated as pilot study in the thesis. A dynamic prosody modification algorithm is developed and tested in different languages varying in linguistic similarity. Performance evaluation is conducted using standard metrics like root mean square error (RMSE) and correlation. Although their acoustic and perceptual features differ as a function of linguistic similarity, some parameters fundamental for emotional expression occurred ...