Novel query analysis and ontology based clustering for data management in hadoop

Abstract

In the current information explosion era, the amount of data stored in the form of text, image, video and audio is enormous and is expected to grow in future. Moreover, the technological innovations in fields like e-libraries and epublishing is increasing the magnitude and usage of digital documents. Thus, the tools that can be used to analyze and discover useful knowledge are in great demand. In particular, the usage of data mining techniques on text documents, also known as intelligent text analysis, text data mining or Knowledge Discovery in Text (KDT) is becoming more popular. The reason behind this popularity is that most of the information (over 80%) is currently stored as text and therefore mining text information has a great commercial potential value. Document clustering helps users to effectively navigate, summarize and organize text documents into meaningful clusters, knowledge that helps to handle huge amount of text extraction. The continuous increase of computational power causes astonishing flow of data. Processing this voluminous data in a reliable and cost effective manner to draw valuable insights and business intelligence is of rapidly growing interest. Difficulties among the processes such storing, analyzing and visualizing the huge amount of datasets can be effectively handled with the big data concepts. Big data analytics can be extract hidden patterns and secret correlations, this kind of business assistances leads a greater successes to the organization. For this reason, big data implementations got to be analyzed and executed as accurately as possible. Bottleneck issues handled in the field of information retrieval are analysis of query and management of data storage. Hadoop is a large scale environment that is supported with larger storage and faster processing. Even though, it suffers from these challenging issues while the number of information requesters is higher. newline