Mining large data

Abstract

Large datasets require special attention to reduce the huge time and space complexity of different mining algorithms when applied over such massive datasets. To overcome these challenges the complete data can be stored newlineand processed in a distributed manner. Most of the mining and learning algorithms in the literature follow a sequential approach and require complete data to be loaded in the memory directly or virtually to generate the mining results which in case of large datasets is almost infeasible. Applying newlinemining or learning process over a large dataset requires a decomposition of the complete process into independent subprocesses. These subprocesses when applied over distributed data should retrieve local information which is further aggregated to generate the global information.