Design and Development of Document Summarization Algorithms for Kannada

Abstract

The fast growth of Internet overwhelmed the people by large amount of information available online. To obtain the data required by theuser from this huge amount of documents text is time consuming and difficult process. So, there is a need of some system which deals with this information overload. newlineThere are various modes of Information Retrieval system - Text Mining, Machine Translation, Text Categorization, Text Summarization etc. In this thesis, an effort has been placed on studying certain aspects of Text Summarization as a solution to the information overload problem. Text Summarization is the process of condensing the original text in the document into shorter form. newlineLiterature shows a very few attempts in generating the summary of documents in Indian languages. The main reason behind this is the complexity involved in developing the resources and tools for Indian languages. In the exhaustive research work of this thesis, somenew ideas have been included to the literature oftext summarization considering Kannada language as the case. newlineIn the first contribution, dataset required for developing text summarization algorithm for Kannada has been developed. The data required for developing Parts-Of-Speech(POS) tagging tool and Named Entity Recognition (NER) tool has been developed. newline In the second contribution,algorithms for generating extractive summary of Kannada document have been developed.Techniques Latent semantic analysis, Clustering, Fuzzy logic and Fused approach has been applied to produce the extractive summary. newlineThe POS tagging tool for Kannada using Conditional Random Field (CRF) has been developed in the third contribution. newlineIn the fourth Contribution,algorithms have been developed to recognize and classify the named entities in Kannada document using CRF and rule-based approach. newlineAn algorithm for creating an abstractive summary using template-based approach for Kannada document has been developedin thefifth Contribution newline newline