Improved extraction mechanism in ETL process for building of a data warehouse

Abstract

Today the Data warehouse plays an important role in decision making, data analysis, and strategic information and so on. Extraction Transformation and Loading (ETL) process is used very popularly in building up of the data warehouse. In todays competitive Business world Mergers and Acquisitions are very common. It requires Extraction, Transformation and Loading of a huge amount of Organizational Data movement. This work proposed with respect to the Time and Space domains by the improvement in the extraction speed and file size. The existing approach is modified to apply both the standard methods of extraction, the Full Extraction and Incremental Extraction Procedures in the ETL Process. Though flat files have been widely used for simplification of the database file and existing systems does not provide any security for the same. In this thesis we are proposing the various security methods to apply over a flat file during the extraction process in ETL. These security methods can be highly recommended for very high sensitive data. Applications of security measures presented in this thesis are effective to reduce the possibility of the data leakage risk. To further reduce the size of the file the use of the delimited flat files rather than fixed length flat files converted from database files is suggested. Consequently results with reducing the extraction time of the data file. Such conversion proposal would certainly improve the speed of the extraction as presented in this thesis work. This work has provided preparatory inputs for the building process of Data warehouse and how ETL contributes to the major role in the building process. As these flat files are created from the database files the problems like redundancy, consistency, and more searching time does not occur and we can make use of the advantages of flat file during extraction process. This work provides security to the flat files before extraction and then it is extracted from the source to the destination system.