Analysis of resource management Schemes using bio inspired algorithm For hadoop cluster in cloud Computing

Abstract

Big data has drawn dynamically more ideas in the field of Information Technology (IT) since IT business is continuously developing. Due to the enhancement in the World Wide Web, it explores a high dimensional data that guides the growth of the business model and creates an incredible trade opening. Since the Big data is more complex and enormous, the regular handling tools such as storing, regulating and data analysing process at a suitable length become a difficult task. newlineBig data can perform various operations like gathering, handling, perceiving and sharing a vast amount of data from different sources. Hadoop, an open-source tool is utilized here to handle the large volume of the data in a cost-efficient and effective way. Here, multiple jobs can be executed in parallel to archive the valid information. Hadoop Map-reduce framework is widely used for processing huge amount of data. MapReduce framework employs the scheduler to execute the jobs based on resource availability in the Hadoop cluster. newlineThe first method aims to propose the effective DataNode assignment technique for resource allocation in the Hadoop MapReduce job. Here, MapReduce jobs are assigned to the task node to perform the map-reduce operation based upon the scheduler. Each node has slots (virtual core) to process a task using the map and reduce operation. Map tasks are executed separately prior to the Reduce task. The different execution order of jobs and different slot configuration in the clusters affect the CPU performance significantly. This reduces the cost of the node effectively and improves the job execution performance in the Hadoop cluster. newline