Named entity recognition For kannada using conditional random fields

Abstract

Natural Language Processing (NLP) has attracted the researchers in recent years newlinebecause of the emerging of automatic text processing technology. NLP is newlinechallenging task in Indian languages due to their morphological richness, newlineambiguities and free word order nature. Very less research work has been done in newlineIndian regional languages and it is growing due to the increase in the usage of newlinemobiles applications. These applications are in general used mainly by common newlinepeople for booking cab to creating official letter in their own native official newlinelanguage. Data processes instantly based on real time scenario. Often generic newlinecorpus looks like a good choice, especially during language translation, sentence newlinesuggestion, opinion mining, sentiment analysis and product analysis. newlineGeneric data collected from Wikipedia, newswires and twitter are always noisy newlineand contain additional data of images, URLs and special symbols. This makes newlineentity recognition task more difficult. Tweets consists only upto a maximum of newline140 characters, and it is smaller than an article abstract or summary. The meaning newlineof the sentence is not clear due to lack of grammatical syntax and use of less newlinenumber of words. In some cases, sentences written are attenuated such that entity newlinerecognition itself becomes a challenging task. Though the entities are newlinefundamental elements used in a sentence to understand its subject. The newlineappearance of entities in the Wikipedia and newswire articles also causes some newlinesyntax variations in the sentence, resulting in limited information on language newlinegrammar. Hence, corpus must undergo pre-processing in the first stage to build newlineefficient entity recognition. This pre-processing must include data cleaning, separation of words as the one or more words are morphologically and newlineorthographically combined together, tokenization and annotation. newlineThough, there are pre-processors available for languages like English, German newlineand Chinese. They fail to perform in Indian languages, due to the difference in newlinesyntax and semantics. Since, language independe