Optimization of association rules in Data Mining using Parallel Approach

Abstract

Data mining is the process of automatic extraction of novel, useful, and understandable patterns in very large datasets. One of the most important problems in Data Mining [1, 6] is discovering association rules. An example of association rule is “30% of all customers who buy jackets and gloves also buy hiking boots”. The association rule problem is to find all such rules whose frequency is greater than some user-specified minimum. One of the key features of previous algorithms developed is that they require multiple passes over the datasets. Over time, as datasets continue to grow inexorably in size and complexity, association rule problem demands more and more computational power. High performance scalable and parallel computing thus becomes crucial for ensuring system scalability and interactivity. This thesis deals with both algorithmic and system aspects for achieving optimization of association rules in data mining using parallel approach. The algorithmic aspect of optimization focuses on the design of efficient, scalable, parallel algorithm for association rules. We assume that the datasets are very large and disk-resident. Computer clusters are commonly used to increase the computational power for solving data mining problems. Clusters can contain homogeneous machines i.e. machines having same CPU speed and memory or they could be collection of heterogeneous machines which would have different CPU speeds and memory. The system aspect deals with scalable implementation on homogeneous and heterogeneous collection of networked workstations. It is commonly observed that more the number of components in a system, the probability of failure increases.