Generative model driven representation learning with discriminative classifier for environmental audio scene and sound event recognition

Abstract

The analysis of sound information is very helpful in multimedia information retrieval, audio surveillance, audio tagging, and forensic applications. Environmental Audio Scene Recognition (EASR) and Sound Event Recognition (SER) are the principle tasks that are related to audio surveillance systems. Environmental Audio Scene Recognition refers to the process of recognizing the context or environment of an audio stream, with applications in devices requiring contextual awareness. Sound Event Recognition aims to recognize the occurrence of a monophonic event in a specific environment. Environmental Audio Scene Recognition and Sound Event Recognition are challenging tasks due to the presence of multiple sound sources, background noises and overlapping or polyphonic contexts. In the environmental audio scene recognition task, the typical duration of an environmental audio scene is long, and it is in the range of few seconds to few tens of seconds. Different scenes of the same class may take different durations. An important issue is that when the data of different classes are more confusable, generative model-based classifiers such as Gaussian Mixture Model (GMM) are not appropriate since a model is built for each class using the samples of that class only. This contributes to the idea of a discriminative model-based approach that recognize the examples of environmental audio scenes. Discriminative model based classifiers such as support vector machines (SVMs) focus on modeling the decision boundaries between classes. Another issue is that SVM can handle fixed dimensional data only. In this thesis, these issues are addressed by proposing a hybrid framework that learns model-driven representations for environmental audio scenes and sound events with the help of generative models newline