An Intelligent Energy Aware Approach for Big Data Storage in Cloud Data Centers

Abstract

The advancement in current technology has lead to the rapid rise in big data applications like E-commerce, scienti c computing, healthcare etc. These applications require enormous computing capabilities such as high end infrastructure, platforms and softwares. Cloud data centers provide these facilities based on pay as you go model, yet raise several challenges which include energy e ciency, scalability, privacy, and storage etc. Among these issues, energy e ciency has turned into an upcoming challenge for executing the big data applications in cloud environment. Energy has become a critical resource in modern computing systems, which presents challenges to the traditional storage systems. The energy consumed by the storage subsystem surpasses all other sub-components present in the server. The disks in high-end servers are responsible for the high power consumption. Hence, the prediction based energy-aware approach is required for an e cient data placement among the disks to power it down for the long duration. Prediction also helps in identifying which data objects need to be replicated. Therefore, an integration of data prediction with placement along with the disk scheduling provides an optimal solution to reduce energy and time consumption. To achieve the set objectives, an extensive literature survey of existing data prediction models and energy e cient storage techniques has been done. But the previous research does not cover all the aspects such as data prediction, data placement including replica management and disk scheduling for big data storage. Therefore, in this work, an intelligent energy aware approach is proposed to reduce storage energy consumption in the cloud environment. Firstly, the storage prediction model has been proposed that generates and customizes the SQL traces to nd the frequency of each query red on the real data streams obtained from the SCATS sensors of Dublin city. Based on the calculated frequency, the future frequency of each query has been predicted using ensemble approach.