Efficient filter based refinement clustering method for data reduction in large data sets

Abstract

The data reduction implies reducing the size of the data set but without losing the integrity exist among them. The data reduction techniques aim to obtain a reduced representation of the data set that is very small in volume, but it maintains the integrity of the original data. Therefore, the data reduction mines on the reduced data set which is efficient and produce the same analytical result. In this work the data reduction is achieved with respect to clustering and with classification and finally the results are analyzed. To group similar objects, clustering is a powerful technique in machine learning. It is popularly known for its efficiency and for discovering unknown knowledge from very large, complex data sets.The data reduction is carried out via clustering in two types. The first type involves three step processes i. Using preprocessing filters ii. Application of the method called selection of attributes iii. Finally, application of the two different clustering algorithms EM and K-Means on the reduced preprocessed data. a. Initially the filters named Normalize and Randomize from attribute and instance levels are uniquely applied on a livestock dataset and the filtered data are recorded. b. Secondly selection of attribute method is applied on the filtered data and reduced data is obtained. newline