Efficient Computing Of Big Data Harmonization

Abstract

By the improvement and expansion of the internet, social media, internet of things and advanced technology in the fields of healthcare, infrastructure, Agriculture, Education, Scientific fields and in Data Analytics, data generation growth augmented exponentially. In the world of exploding data, storage and speed become the burning issues. Big Data was in existence since long back but due to hype of social media usage it is well-known now. Cost-effective and innovative methodology to process information which can be used for good decision making is in demand. To manage, process and to analyze Big Data both academia and industry work together for cost effective solutions. Big Data harmonization is the process of providing a single platform to all heterogeneous data and variety of data. Extraction, transformation and loading is the essential step in the process of data warehouse. Data harmonization is the alternate name for the data warehouse to provide the common level of granularity. It is the base platform to work upon for OLAP servers and data analytics. Computing of Big Data OLAP requires lot of challenges like scaling of data, speed of processing, storage of data, query performance and lot of others. Mainly Roll up, Drill down, Slice and Dice operations are performed on data. In Big Data Era ROLAP (Relational Online Analytical Processing) or HOLAP (Hybrid Online Analytical Processing) takes more space due to costly joint operation and takes more evolution time to process query. In this thesis MOLAP (Multidimensional OLAP) is adopted. Main focus is to work up on two challenges of Big Data as storage and velocity over OLAP. To work effectively and in parallel manner to deal with volume of Big Data we implemented distributed environment. Author has proposed and implemented a technique name as OOHI (OLAP on Hadoop by Indexing) that offer simplified and efficient multidimensional model. Overall work of OOHI is divided into Data Loading Module, Data Storage Module, Dimension Encoding Module, Dimension traversal M