Video Representation and Semantic Information Extraction using Ontology based Approaches

Abstract

Due to the perceptual information about actions and events contained in video data, newlinewhich is unstructured and hard to quantify, there is a semantic gap between computer newlineinterpretation and human understanding. Consequently, the majority of surveillance newlinesystems rely on a human agent to watch and identify suspicious situations. However, newlinehuman agents are expensive and frequently afflicted by weariness and inattention, which newlinemay result in accidents. Additionally, storing and evaluating visual data is one of the newlinemost significant challenges. However, it can be resolved by encoding data in a machinereadable newlineand machine-interpretable format, thereby facilitating semantic storage and newlineretrieval. Moreover, the multifaceted benefits inspire researchers to investigate video newlinerepresentation in machine-readable and machine-interpretable formats for ease of storage, newlinereasoning, and analysis in order to obtain semantic information. Various metadata newlineleveraging Semantic Technologies for extracting and representing salient video information newlinein a structured format are demonstrated in this thesis work which further helps newlineto obtain semantic information of complicated events/activities/unusual events. This newlinerepresentation requires knowledge extraction, ontology engineering, representation in newlinesymbolic form, reasoning, and retrieval via querying. newlineThe first section of the thesis discusses mid-level feature extraction, which helps for newlineidentifying object tracking information. It further utilized to identify crowd behavior of newlineloitering and physical separation by employing a dynamic threshold proportional to the newlineobject size. In addition, a method is proposed to measure the real physical distance using newlinethe human height as a reference. The proposed technique performance has been benchmarked newlineand validated with MOT datasets, PETS 2006 datasets, and UCF datasets, all newlineof which indicate encouraging results. A full video representation framework is then newlineproposed and demonstrated for identification of suspicious behavior in a parking space newlinein the preceding