An optimized analytical approach for Big Data processing in Cloud Computing Environment

Abstract

In analytics, the speed at which Big Data (BD) is processed has huge importance. For BD analysis, Hadoop is one of the popular frameworks. Map Reduce (MR) is a hugely accepted parallel computing system for larger-scale Data Processing (DP). While processing BD, chief performance metrics in a parallel computing framework is job Execution Time (ET) along with cluster throughput. Performance is severely impacted by the straggler machines while processing BD. The system that consumes prolonged time for completing the task is termed a straggler machine. Moreover, in a parallel computing framework, one single slow task or slow node can severely affect the performance of overall job execution. newlineBy merely backing up these slow-running tasks on substitute machines, the most common approach to deal with stragglers is Speculative Execution (SE). The researchers proposed many SE strategies; however, a few drawbacks were included: (A) average progress rate was used which might be misleading to detect the slow task, (B) Phase wise estimation of Remaining Time of the slow-running task and Processing Speed (PS) (C) calculation of slow task s backup time (D) to rerun the task, detect the proper backup node. newlineIn the Cloud Computing (CC) environment, handling the issue of SE would be more challenging in which clusters are heterogeneous as Hadoop believes that nodes are homogeneous. newlineTo enhance BD processing performance in the case of slow-running tasks in the heterogeneous cloud-based Hadoop Cluster (HC), the proposed Speculative Execution strategy renamed Hadoop Supreme Rate Performance (Hadoop-SRP) can be utilized. For precisely and swiftly identifying stragglers the following are wielded (1) To estimate the PS in the Map/Shuffle/Reduce phase considering parameters process speed and process bandwidth, the Modified EWMA methodology is used (2) Calculate the slow tasks RT (3) calculation of slow task s back up (4) By using the BM-LOA (Behavioral Model-based Lion Optimization Algorithm), re-run the slow task on the identified node. newlineB