Meta data conceptual mining model using analysis of bilateral intelligence for effective text clustering

Abstract

Data mining from organizational data for extracting hidden knowledge is a growing field of study, which forms new study groups named knowledge discovery in database. Engineering applications of data mining which include, web mining, network mining, image mining and multimedia mining in general and text mining in particular. Predicting business flow, finding recent trends, sales and markets monitoring, forecasts, competition monitoring, expenditures control, revenues and administrative processes, finding volcanoes on Venus, Earth geophysics, earthquake photography from space, atmospheric science are some of the real world applications of data mining. Around 80% of data in the world are stored in unstructured textual format. Hence, the text mining and document clustering are major research areas in the past few years. Designing a effective and novel text mining requires high dimensionality, dynamic methodology, fast information access, specialized knowledge extraction from very huge data sets. Variety of approaches are proposed in the literature and these approaches vary with a range of tranditional k-means algorithm, term based methods, pattern taxonomy model and rank based methods. Existing text mining methods has many pitfalls like slow processing, lesser scalability, unable to solve conceptual problems like synonomy and polysemy. Therefore, this thesis focused text mining, proposed conceptual analysis and concentrated to solve conceptual problem like synonomy. Initially, this research work proposed metadata conceptual mining model for effective text mining. The proposed work executes in two phases of manipulation, which are training phase and testing phase. Initially in the pre-processing stage, the di-grams such as in, as, it; and tri-grams such as are, for, ing are removed from the documents. The proposed work created a data structure called, Significant Term List (STL) for each category of documents. A list of keywords on each domain of study and for each field of study is added in the concern STL.