Design and development of topic detection technique from cross media data

Abstract

Social networks such as Facebook, and Twitter, content-sharing platforms such as Flickr, and video-sharing websites such as YouTube generate a huge amount of cross-media data. This repository of huge cross-media or multimedia data challenges the users to search for their intended content. So, we need to organize this huge cross-media data to facilitate the users to reach their intended content. One such method of organizing this cross-media data is topic detection and tracking (TDT). TDT tries to find the significant content from data at the topic level, which helps users better understand different events occurring on different social media platforms. Video data on social media platforms is indirectly connected through heterogeneous interactions among users such as uploading, viewing, and commenting. A typical social video collection can be converted into a video social network by considering these heterogeneous interactions, which helps users in identifying the important aspects of a topic. The key challenge afterwards remains the utilization of this network for topic identification. In this thesis, different techniques such topic detection from cross-media, crossmedia topic structurization techniques i.e. Web video clustering and influential node detection, and a technique for video keyframe extraction using multi-visual feature fusion have been proposed to identify the most representative keyframes for the topics. First of all, to identify most representative key frames from social media videos, a technique based on multi-visual feature fusion has been proposed. Web video clustering and influential node detection techniques have proposed based on heterogeneous social media interactions. For Cross-media topic detection, a technique based on deep transfer learning and latent Dirichlet allocation has been proposed. This technique uses multi-modal data. Another technique based on Bidirectional Encoder Representations from Transformers (BERT) for single media data has been proposed. newline