Design and Analysis of Similarity Measures for Text Document Clustering

Abstract

The advent of computers and mobile devices aroused a revolution in the realm of newlineinformation technology. Digitization of almost everything around us has led to the newlinegeneration of various forms of data types including structured and unstructured. newlineHowever, the investigations carried out by researchers have proved that a considerable newlineportion of data is in text form. Therefore, it is significant to explore and analyze this newlineunstructured data to extract relevant information and transform it into high-quality newlinestructured information. An efficient organization of this kind of data has become newlineanother necessity for today s vast repositories. One of the techniques that is beneficial newlinein this case is clustering, where its primary purpose is to automatically assess the similarity between the text documents and accordingly group them in a way that the ones together possess a strong internal similarity. The usefulness of the text document newlineclustering can be viewed in many areas, for instance, optimization of the indexing and newlineretrieval operations for systematic browsing, organization of the results obtained from newlinesearch engines, summarization of the corpus of documents, and categorization of the newlinecustomer comments for collaborative recommendations. newlineAbove all, in the text document clustering process, the similarity and dissimilarity newline(distance) measures are considered as the heart of every clustering algorithm. Basically, newlinethe success or failure of a clustering algorithm is highly related to the potentiality of the measure that is employed to evaluate how much the pair of documents are similar or far apart from each other. To this end, determining appropriate measures results in the achievement of accurate outcomes. newlineIn this thesis, the work begins with the study and performance evaluation of various similarity and dissimilarity measures that are suitable to be used by clustering algorithms for grouping the text documents.